155

I do not have much experience in table design. My goal is to create one or more product tables that meet the requirements below:

  • Support many kinds of products (TV, Phone, PC, ...). Each kind of product has a different set of parameters, like:

    • Phone will have Color, Size, Weight, OS...

    • PC will have CPU, HDD, RAM...

  • The set of parameters must be dynamic. You can add or edit any parameter you like.

How can I meet these requirements without a separate table for each kind of product?

Air
  • 7,120
  • 2
  • 47
  • 78
StoneHeart
  • 14,122
  • 31
  • 65
  • 83

4 Answers4

258

You have at least these five options for modeling the type hierarchy you describe:

  • Single Table Inheritance: one table for all Product types, with enough columns to store all attributes of all types. This means a lot of columns, most of which are NULL on any given row.

  • Class Table Inheritance: one table for Products, storing attributes common to all product types. Then one table per product type, storing attributes specific to that product type.

  • Concrete Table Inheritance: no table for common Products attributes. Instead, one table per product type, storing both common product attributes, and product-specific attributes.

  • Serialized LOB: One table for Products, storing attributes common to all product types. One extra column stores a BLOB of semi-structured data, in XML, YAML, JSON, or some other format. This BLOB allows you to store the attributes specific to each product type. You can use fancy Design Patterns to describe this, such as Facade and Memento. But regardless you have a blob of attributes that can't be easily queried within SQL; you have to fetch the whole blob back to the application and sort it out there.

  • Entity-Attribute-Value: One table for Products, and one table that pivots attributes to rows, instead of columns. EAV is not a valid design with respect to the relational paradigm, but many people use it anyway. This is the "Properties Pattern" mentioned by another answer. See other questions with the eav tag on StackOverflow for some of the pitfalls.

I have written more about this in a presentation, Extensible Data Modeling.

Additional thoughts about EAV: Although many people seem to favor EAV, I don't. It seems like the most flexible solution, and therefore the best. However, keep in mind the adage TANSTAAFL. Here are some of the disadvantages of EAV:

  • No way to make a column mandatory (equivalent of NOT NULL).
  • No way to use SQL data types to validate entries.
  • No way to ensure that attribute names are spelled consistently.
  • No way to put a foreign key on the values of any given attribute, e.g. for a lookup table.
  • Fetching results in a conventional tabular layout is complex and expensive, because to get attributes from multiple rows you need to do JOIN for each attribute.

The degree of flexibility EAV gives you requires sacrifices in other areas, probably making your code as complex (or worse) than it would have been to solve the original problem in a more conventional way.

And in most cases, it's unnecessary to have that degree of flexibility. In the OP's question about product types, it's much simpler to create a table per product type for product-specific attributes, so you have some consistent structure enforced at least for entries of the same product type.

I'd use EAV only if every row must be permitted to potentially have a distinct set of attributes. When you have a finite set of product types, EAV is overkill. Class Table Inheritance would be my first choice.

Update 2019: The more I see people using JSON as a solution for the "many custom attributes" problem, the less I like that solution. It makes queries too complex, even when using special JSON functions to support them. It takes a lot more storage space to store JSON documents, versus storing in normal rows and columns.

Basically, none of these solutions are easy or efficient in a relational database. The whole idea of having "variable attributes" is fundamentally at odds with relational theory.

What it comes down to is that you have to choose one of the solutions based on which is the least bad for your app. Therefore you need to know how you're going to query the data before you choose a database design. There's no way to choose one solution that is "best" because any of the solutions might be best for a given application.

Bill Karwin
  • 462,430
  • 80
  • 609
  • 762
  • 11
    @HimalayaGarg Option "4.5" really is the opposite of the whole point of Bill's post. – user3308043 Aug 29 '14 at 19:28
  • 2
    Unlike MySQL, SQL Server has extensive support for XML, XPath and XQuery. So for users of SQL Server, the best option would be to store extra attributes in a column of XML type (option 4). This way you DON'T have to "fetch the whole blob back to the application and sort it out there." You can even create indexes on XML columns in SQL Server. – Delphi.Boy Oct 02 '14 at 06:48
  • 1
    @Delphi.Boy, great tip! [Oracle also supports indexing XML](http://docs.oracle.com/cd/B28359_01/appdev.111/b28369/xdb_indexing.htm#CHDJECDA). [IBM DB2 also supports indexing XML](http://www.ibm.com/developerworks/data/library/techarticle/dm-0611nicola/). – Bill Karwin Oct 02 '14 at 15:25
  • 2
    I prefer Serialized LOB for my case. But is it suitable for ORM? I use EF. – Mahmood Jenami Mar 22 '15 at 14:10
  • @user2741577, sure, but you'll probably have to write custom code to unpack the fields of unstructured data form the LOB and apply them to each entity field of your ORM object. I don't know EF, but I suppose you could create a base ORM class that does this. You need to keep track of which fields came from concrete fields of the database row, and which fields came from fields of the LOB, so you can re-form a LOB when it is time to save the object. – Bill Karwin Mar 22 '15 at 17:32
  • @Bill Karwin so what is the best way what you prefer ? – babak faghihian Sep 23 '15 at 13:04
  • @babakfaghihian, each situation must be considered individually. Depending on the types of queries we will make against the data, any solution might be best for that occasion. I don't think any of these solutions are better than the other, I think of them all as tools in my tool belt, each with their advantages for different circumstances. – Bill Karwin Sep 23 '15 at 14:45
  • 1
    Just a small update. Today the most recent versions of both mysql and postgresql, can query data within a JSON field. – Benjamin Hammer Nørgaard Mar 03 '16 at 08:12
  • 1
    I have found a good [article](https://scotch.io/tutorials/working-with-json-in-mysql) on mysql JSON datatype. It looks better and easier than EAV method but I don' t know about its performance – Uzair Ali Mar 30 '18 at 19:20
  • @BillKarwin - if you had 2 or 3 products that needed association tables but the association tables were the same schema, does that sway more towards STI vs CTI or still the same in terms of it depending on what queries you're doing? I'll have queries where I need specific product data from specific products and base products that also require association to those 2 or 3 product association tables but I'll also have queries where I want to just get the base product info from 2 or 3 types which I would assume would require a product_type enum column or product_type_id column? – meder omuraliev Dec 03 '18 at 04:25
  • @mederomuraliev If it's only 2 or 3 products, and not likely to increase, I would lean toward STI. If you want more flexibility to add new product types in the future without altering the base table, prefer CTI. – Bill Karwin Dec 03 '18 at 07:24
  • @BillKarwin even if it meant 15-20 additional "unique" columns that only apply to the 2 or 3 product types? And you would control that / make it more efficient by selecting all the rows you actually need, versus selecting all obviously. (among other things). – meder omuraliev Dec 03 '18 at 14:36
  • @mederomuraliev 15 or 20 is a finite number of columns. By using STI or CTI, you still can use data types and constraints. If you use STI, you lose the option of `NOT NULL` but you can use other constraints. In this way, both STI and CTI are better than using EAV or JSON. – Bill Karwin Dec 03 '18 at 15:26
14

@StoneHeart

I would go here with EAV and MVC all the way.

@Bill Karvin

Here are some of the disadvantages of EAV:

  • No way to make a column mandatory (equivalent of NOT NULL).
  • No way to use SQL data types to validate entries.
  • No way to ensure that attribute names are spelled consistently.
  • No way to put a foreign key on the values of any given attribute, e.g. for a lookup table.

All those things that you have mentioned here:

  • data validation
  • attribute names spelling validation
  • mandatory columns/fields
  • handling the destruction of dependent attributes

in my opinion don't belong in a database at all because none of databases are capable of handling those interactions and requirements on a proper level as a programming language of an application does.

In my opinion using a database in this way is like using a rock to hammer a nail. You can do it with a rock but aren't you suppose to use a hammer which is more precise and specifically designed for this sort of activity ?

Fetching results in a conventional tabular layout is complex and expensive, because to get attributes from multiple rows you need to do JOIN for each attribute.

This problem can be solved by making few queries on partial data and processing them into tabular layout with your application. Even if you have 600GB of product data you can process it in batches if you require data from every single row in this table.

Going further If you would like to improve the performance of the queries you can select certain operations like for e.g. reporting or global text search and prepare for them index tables which would store required data and would be regenerated periodically, lets say every 30 minutes.

You don't even need to be concerned with the cost of extra data storage because it gets cheaper and cheaper every day.

If you would still be concerned with performance of operations done by the application, you can always use Erlang, C++, Go Language to pre-process the data and later on just process the optimised data further in your main app.

John Kugelman
  • 307,513
  • 65
  • 473
  • 519
Pawel Barcik
  • 511
  • 6
  • 9
  • `you can always use Erlang, C++, Go Language to pre-process the data` What did you mean? Instead of DB, use Go lang? Could you please elaborate on that? – Green Nov 06 '16 at 08:15
  • 1
    I totally agree. EAV is a way to go, especially if you need level of flexiblity which would allow you to add new kind of products and parameters without db schema changes, I mean live in production via your applicaion. Been there, done that. Worked for me. About slow queries... did anyone here has ever heard about caches? ;) – pawel.kalisz Dec 01 '16 at 07:35
  • @Green I've edited the last paragraph to make it more clear, but it's about passing your raw EAV data to a process in a language which can deal with data transformations, lookups in a tree structure or any basic map reduce operations really quickly and in a memory efficient way. The specifics here would depend on what needs to be optimised – Pawel Barcik Feb 07 '17 at 16:44
7

If I use Class Table Inheritance meaning:

one table for Products, storing attributes common to all product types. Then one table per product type, storing attributes specific to that product type. -Bill Karwin

Which I like the best of Bill Karwin's Suggestions.. I can kind of foresee one drawback, which I will try to explain how to keep from becoming a problem.

What contingency plan should I have in place when an attribute that is only common to 1 type, then becomes common to 2, then 3, etc?

For example: (this is just an example, not my real issue)

If we sell furniture, we might sell chairs, lamps, sofas, TVs, etc. The TV type might be the only type we carry that has a power consumption. So I would put the power_consumption attribute on the tv_type_table. But then we start to carry Home theater systems which also have a power_consumption property. OK its just one other product so I'll add this field to the stereo_type_table as well since that is probably easiest at this point. But over time as we start to carry more and more electronics, we realize that power_consumption is broad enough that it should be in the main_product_table. What should I do now?

Add the field to the main_product_table. Write a script to loop through the electronics and put the correct value from each type_table to the main_product_table. Then drop that column from each type_table.

Now If I was always using the same GetProductData class to interact with the database to pull the product info; then if any changes in code now need refactoring, they should be to that Class only.

JD Isaacks
  • 51,154
  • 89
  • 267
  • 413
4

You can have a Product table and a separate ProductAdditionInfo table with 3 columns: product ID, additional info name, additional info value. If color is used by many but not all kinds of Products you could have it be a nullable column in the Product table, or just put it in ProductAdditionalInfo.

This approach is not a traditional technique for a relational database, but I have seen it used a lot in practice. It can be flexible and have good performance.

Steve Yegge calls this the Properties pattern and wrote a long post about using it.

RossFabricant
  • 11,514
  • 3
  • 39
  • 51
  • 4
    The Properties Pattern is just Entity-Attribute-Value by another name. It is used widely, but storing it in a relational database breaks rules of normalization. – Bill Karwin Mar 30 '09 at 02:44
  • 2
    To be honest, when I read the description of EAV in @Bills answer I did not quite understand what he was explaining. But when you said `3 columns: product ID, additional info name, additional info value` I understood the concept. And I have actually done this before, and ran into problems. However, I don't recall at the moment what those problems were. – JD Isaacks Sep 16 '10 at 18:55
  • 1
    @JDIsaacks In this pattern, a common problem is that we don't know how many JOINs do we need to fetch all attributes. – Omid Dec 03 '13 at 08:00