[blf@Logging /~]:

April 9, 2008

[mailist] How to handling large volumes of data on PostgreSQL?

Filed under: database, PostgreSQL — blowfisher @ 7:14 pm

mailing list: pgsql-admin.postgresql.org

from: Johann Spies

..loaded about 4,900,000,000 in one of two tables with 7200684 in the second table in database ‘firewall’, built one index using one date-field (which took a few days) and used that index to copy about 3,800,000,000 of those records from the first to a third table, deleted those copied record from the first table and dropped the third table.
This took about a week on a 2xCPU quadcore server with 8Gb RAM..

Table paritioning is need.

distribute tables across different disks through tablespaces.Tweak the shared buffers and work_mem settings.

RAID5/6 are very,very slow when it comes to small disk *writes*.

At least a hardware RAID controller with RAID 0 or 10 should be used, with 10krpm or 15krpm drives. SAS preferred.

as on SATA the only quick disks are Western Digital Raptor.

look at a view called pg_stat_activity. Do: select * from pg_stat_activity;

1 Comment »

  1. 最近看到 文章,Skype 使用 PostgreSQL 支持 10亿账户。

    http://www.dbanotes.net/arch/skype_postgresql.html

    [
    今天看到 Skype Plans for PostgreSQL to Scale to 1 Billion Users 这个帖子,对 PostgreSQL 在大型网站应用上的部署算是有了一点了解。
    ]

    Comment by likuku — April 9, 2008 @ 10:54 pm

RSS feed for comments on this post. TrackBack URL

Leave a comment

You must be logged in to post a comment.

www.blowfisher.net  |  Powered by WP