david-adam · October 8, 2012 11:34
diff --git a/bigdata.ini b/bigdata.ini
 /*
 1. Do not store the comments as content objects is a no-brainer
 - either use ezcomments, offsite comments (disqus etc) or a custom implementation

 2. Usage of async publishing is probably mandatory. With such high-frequency of content publishing you want as little deadlocks happening on your content as possible, which means keeping transactions fast and limiting their concurrency

 3. nosql / other storage engines: depends on project's deadline.
 eZP 5.0 is still quite young, and you will not get any of those from Engineering until 5.1 at least. So you will need to build your own, which is going to be VERY time consuming. I would rather create some dedicated cache-storing layer that stuffs things in memcache (maybe using zetacomp's caching component) and use ith with 4.x.
 Consider also that currently ezp5 has LESS caching built-in than ezp4, and it might use more queries per page.

 5. design your caches properly: you do not want to invalidate 300k objects view cache when you publish one (because of too many object relations or crazy smartviewcache rules)

 6. design your content properly: do not create 5M objects in a single folder node

 7. if most accesses to the site are from anon users, and you have a lot of trafffic, do ESI big-style. Get ezaccellerator2, pay Novactive for a couple of days of consulting to explain to you how they did esi-blocks right and replaced completely cache-blocks (start with http://www.slideshare.net/morelseb/varnish-e-zpublish)

 8. store sessions in memcache, not db, as having 500 editors connected would keep the db busy with their session data

 9. buy a honking BIG db server, properly tune it. It might work AGAINST eZ support policy, but you should really get the best out of mysql technology:
 - use at least mysql 5.5
 - tune replication, buffer pool, transaction isolation level, query cache, disk io. Make sure temp tables are stored in RAM
 - getting an ssd card might be a good investment even if it costs money
 - use proper monitoring / analysis tools to understand what the db is doing, what can be optimized (are there deadlocks when a cache-block expires? etc)
 - it could be worth playing around with percona or other variants
 - table partitioning might be an excellent option, but it involves a lot of testing:
  + we have no experience with it (a few guys in the forums mentioned it, but never came back with numbers)
  + you should partition your tables in a way that the most frequent selects affect 1 partition at at time only. depending on your fetches, this might mean partitioning on one column or another (section, class, substring(path) etc)

 10. use ezperformancelogger to help analyze/troubleshoot performances (http://www.slideshare.net/gggeek/ez-performance-measurement)
 */
	/*
	1. Do not store the comments as content objects is a no-brainer
	- either use ezcomments, offsite comments (disqus etc) or a custom implementation

	2. Usage of async publishing is probably mandatory. With such high-frequency of content publishing you want as little deadlocks happening on your content as possible, which means keeping transactions fast and limiting their concurrency

	3. nosql / other storage engines: depends on project's deadline.
	eZP 5.0 is still quite young, and you will not get any of those from Engineering until 5.1 at least. So you will need to build your own, which is going to be VERY time consuming. I would rather create some dedicated cache-storing layer that stuffs things in memcache (maybe using zetacomp's caching component) and use ith with 4.x.
	Consider also that currently ezp5 has LESS caching built-in than ezp4, and it might use more queries per page.

	5. design your caches properly: you do not want to invalidate 300k objects view cache when you publish one (because of too many object relations or crazy smartviewcache rules)

	6. design your content properly: do not create 5M objects in a single folder node

	7. if most accesses to the site are from anon users, and you have a lot of trafffic, do ESI big-style. Get ezaccellerator2, pay Novactive for a couple of days of consulting to explain to you how they did esi-blocks right and replaced completely cache-blocks (start with http://www.slideshare.net/morelseb/varnish-e-zpublish)

	8. store sessions in memcache, not db, as having 500 editors connected would keep the db busy with their session data

	9. buy a honking BIG db server, properly tune it. It might work AGAINST eZ support policy, but you should really get the best out of mysql technology:
	- use at least mysql 5.5
	- tune replication, buffer pool, transaction isolation level, query cache, disk io. Make sure temp tables are stored in RAM
	- getting an ssd card might be a good investment even if it costs money
	- use proper monitoring / analysis tools to understand what the db is doing, what can be optimized (are there deadlocks when a cache-block expires? etc)
	- it could be worth playing around with percona or other variants
	- table partitioning might be an excellent option, but it involves a lot of testing:
	+ we have no experience with it (a few guys in the forums mentioned it, but never came back with numbers)
	+ you should partition your tables in a way that the most frequent selects affect 1 partition at at time only. depending on your fetches, this might mean partitioning on one column or another (section, class, substring(path) etc)

	10. use ezperformancelogger to help analyze/troubleshoot performances (http://www.slideshare.net/gggeek/ez-performance-measurement)
	*/