[mailist] How to handling large volumes of data on PostgreSQL?
mailing list: pgsql-admin.postgresql.org
from: Johann Spies
..loaded about 4,900,000,000 in one of two tables with 7200684 in the second table in database ‘firewall’, built one index using one date-field (which took a few days) and used that index to copy about 3,800,000,000 of those records from the first to a third table, deleted those copied record from the first table and dropped the third table.
This took about a week on a 2xCPU quadcore server with 8Gb RAM..—
Table paritioning is need.
distribute tables across different disks through tablespaces.Tweak the shared buffers and work_mem settings.
RAID5/6 are very,very slow when it comes to small disk *writes*.
At least a hardware RAID controller with RAID 0 or 10 should be used, with 10krpm or 15krpm drives. SAS preferred.
as on SATA the only quick disks are Western Digital Raptor.
look at a view called pg_stat_activity. Do: select * from pg_stat_activity;

最近看到 文章,Skype 使用 PostgreSQL 支持 10亿账户。
http://www.dbanotes.net/arch/skype_postgresql.html
[
今天看到 Skype Plans for PostgreSQL to Scale to 1 Billion Users 这个帖子,对 PostgreSQL 在大型网站应用上的部署算是有了一点了解。
]
Comment by likuku — April 9, 2008 @ 10:54 pm