[blf@Logging /~]:

Problems with PostgreSQL on Multi-core Systems with Multi-Terabyte Data.

– by Jignesh K. Shah

PDF: http://www.blowfisher.net/pubs/pgcon_problems-JigneshK.Shah.pdf

Current Market Trends in Systems

• Quad-core sockets are current market standards
> Also 8-core sockets available now and could become a standard in next couple of year

• Most common rack servers now have two sockets
> 8-core (or more ) systems are the norm with trend going to 12-16 core systems soon

• Most Servers have internal drives > 146 GB
> Denser in capacity, smaller in size but essentially same or lower speed
> More denser in case of SATA-II drives

Current Market Trends in Software

• Software (including Operating Systems) have yet to fully catch up with multi-core systems
> “tar” still single process utility

• Horizontal Scaling helps a lot but not a good clean solution for multi-core systems

• Virtualization is the new buzzword for Consolidations
> Hides the fact that the software is not able to fully capitalize the extra cores

• Research being done on new paradigms
> Complexity of parallelized software is huge

Current Market Trends in Data

• 12 years ago, a 20GB data warehouse was considered a big database

• Now everybody talks about 200GB-5TB databases

• Some 2005 Survey numbers:
> Top OLTP DB sizes = 5,973 GB to 23,101 GB
> Top DW DB Sizes = 17,685 GB to 100,386 GB
> Source http://www.wintercorp.com/VLDB/2005_TopTen_Survey/TopTenWinners_2005.asp

• Some 2007 Survey numbers:
> Top DB sizes = 20+ TB to 220 TB ( 6+ PB on tape)
> Source http://www.businessintelligencelowdown.com/2007/02/top_10_largest.html