[08-08-30] @Links
[08-08-30] @Links:
August 30, 2008
August 28, 2008
OpenSQL Camp 2008: a community-organized database conference
From announcing-opensql-camp-2008:
Key facts:
- It is of, by and for the community (you).
- At this event, all open-source databases are created equal. We’ll learn together and grow together.
It’s a combination conference and hackathon. - It’s free.
- It is Friday night Nov 14, 18:00 through Sun the 16th at 18:00 in Charlottesville, Virginia USA in a very cool location.
- The website, where all details will be posted: http://opensqlcamp.pbwiki.com/
- The mailing group, where details will be discussed, decided and arranged: http://groups.google.com/group/opensqlcamp
- Date, time and place are confirmed and will not change. You can make your travel plans now. There’s travel information on the wiki.
- We already have some great speakers who have offered to give great talks. These and other details will show up on the wiki as they’re finalized.
- Sponsors are needed. See the wiki.
August 1, 2008
BSD Hammer streaming 的一些信息
看样子有点像 Linux 下的 DRBD
two disks can be synchronized over a network link of any speed or reliability - it can be restarted and immediately begin where it left off, and the amount of bandwidth used can be controlled…
参考链接:
Hammer streaming
HAMMER Update 31-Jul-2008 - Streaming Mirroring on HEAD
July 31, 2008
[08-07-31] @Links
[08-07-31] @Links:
SSD / SATA 阵列性能对比
SSD vs. SATA RAID: A performance benchmark
Microsoft Backs Apache and Open Source
Microsoft announced their sponsorship of “The Apache Software Foundation”,
joining Google and Yahoo! at Platinum level.
KDE 4.1 Release
2008年7月29日 - KDE 4.1 发布
WinSCP: 4.1.6 released
2008-07-30: WinSCP: 4.1.6 released
June 28, 2008
[2008-6-27] Bill Gates retires today
June 26, 2008
June 24, 2008
Symbian 要开源了
可能由 Symbian 基金会 管理
采用的许可证: Eclipse Public License
Symbian Foundation:
Industry leaders to unify the Symbian mobile platform and set it free
Foundation to be established to provide royalty-free open platform and accelerate innovation
相关的链接：
http://symbianfoundation.org
http://www.symbian.com
Nokia buys Symbian, will open Symbian OS
NetBSD 许可协议删除了#3 #4 条款
NetBSD 转向使用两条款 BSD 协议，#3 #4 关于广告支持的条款已从协议文本中删除。
http://mail-index.netbsd.org/…../msg000030.html
另，看一个有趣的对比：
June 21, 2008
说说 Firefox 3.0 的外观，到底咋样？
功能上的增强，不多说了。(地址栏 bookmarks 实时检索，很出彩! )
至于外观，还期待能进一步 “改进” 。。
http://tech.sina.com.cn/s/…/13172130234.shtml
http://www.firefox.net.cn/…?t=22099
June 19, 2008
June 14, 2008
June 10, 2008
Google Trends 三则
June 5, 2008
[PostgreSQL] pgAdmin III v1.8.4 released
2008-06-05 - pgAdmin III v1.8.4 released.
June 4, 2008
微软”跟随战略”之：MS版 memcached - “Velocity”
http://blogs.msdn.com/velocity/
Velocity is intended to provide distributed caching (in memory) for all .NET applications – from enterprise scale to web-scale. We believe that there are many applications that need a distributed caching mechanism, and that there is, therefore, a need for distributed caching as a core part of the .NET platform. We expect to have more integrated support for this functionality with other parts of the .NET platform in our upcoming releases.
[CVS] FreeBSD begins switch to Subversion
FreeBSD 源代码管理系统 开始从 CVS 转向 Subversion 。
3 June，2008:
The FreeBSD Project has begun the switch of its source code management system from CVS to Subversion. At this point in time, FreeBSD’s developers are making changes to the base system in the Subversion repository. We have a replication system in place that exports our work to the legacy CVS tree on a continuous basis.
People who are using our extensive CVS based distribution network (including anoncvs, CVSup, cvsweb, ftp) will not be interrupted by our work-in-progress. We are committed to maintaining the existing CVS based distribution system for at least the support lifetime of all existing “stable” branches. Security and errata patches will continue to be made available in their usual CVS locations.
We expect to make our Subversion based source tree and other supporting infrastructure public soon. There will be new mailing lists to subscribe to if you wish to receive Subversion commit notifications.
Our ports, doc and www trees are not affected at this time. A separate decision will be made regarding those CVS repositories soon.
Many people have contributed to the effort, but we wish to thank Michael Haggerty and the cvs2svn project developers for their assistance with cvs2svn. Peter Wemm spent several weeks of Yahoo!’s time repairing the CVS tree, preparing for and performing the conversion, and the configuration of the Subversion infrastructure. Yahoo! donated the server hardware.
来源：http://www.freebsd.org/news/newsflash.html#event20080603:01
相关：http://blogs.freebsdish.org/ivoras/2008/06/01/freebsd-on-subversion/
有趣的东东，Goosh.org
Goosh.org: a Unix-like Shell For Google
Neat idea:
goosh.org - the unofficial google shell. This google-interface behaves similar to a unix-shell.
You type commands and the results are shown on this page.— by Vivek
May 31, 2008
[HACKERS] Core team statement on replication in PostgreSQL (备查)
built-in replication in PostgreSQL 看来有戏了，留个记号在此备查！
* From: Tom Lane
* To: pgsql-hackers(at)postgreSQL(dot)org
* Subject: Core team statement on replication in PostgreSQL
* Date: Thu, 29 May 2008 10:12:55 -0400
* Message-id: <26529(dot)1212070375(at)sss(dot)pgh(dot)pa(dot)us>
http://archives.postgresql.org/pgsql-hackers/2008-05/msg00913.php
The Postgres core team met at PGCon to discuss a few issues, the largest
of which is the need for simple, built-in replication for PostgreSQL.
Historically the project policy has been to avoid putting replication
into core PostgreSQL, so as to leave room for development of competing
solutions, recognizing that there is no “one size fits all” replication
solution. However, it is becoming clear that this policy is hindering
acceptance of PostgreSQL to too great an extent, compared to the benefit
it offers to the add-on replication projects. Users who might consider
PostgreSQL are choosing other database systems because our existing
replication options are too complex to install and use for simple cases.
In practice, simple asynchronous single-master-multiple-slave
replication covers a respectable fraction of use cases, so we have
concluded that we should allow such a feature to be included in the core
project. We emphasize that this is not meant to prevent continued
development of add-on replication projects that cover more complex use
cases.We believe that the most appropriate base technology for this is
probably real-time WAL log shipping, as was demoed by NTT OSS at PGCon.
We hope that such a feature can be completed for 8.4. Ideally this
would be coupled with the ability to execute read-only queries on the
slave servers, but we see technical difficulties that might prevent that
from being completed before 8.5 or even further out. (The big problem
is that long-running slave-side queries might still need tuples that are
vacuumable on the master, and so replication of vacuuming actions would
cause the slave’s queries to deliver wrong answers.)Again, this will not replace Slony, pgPool, Continuent, Londiste, or
other systems for many users, as it will be not be highly scalable nor
support long-distance replication nor replicating less than an entire
installation. But it is time to include a simple, reliable basic
replication feature in the core system.—regards, tom lane
May 26, 2008
[Jignesh K. Shah] Problems with PG on Multi-Cores with Multi-TB Data
Problems with PostgreSQL on Multi-core Systems with Multi-Terabyte Data.
– by Jignesh K. Shah
PDF: http://www.blowfisher.net/pubs/pgcon_problems-JigneshK.Shah.pdf
Current Market Trends in Systems
• Quad-core sockets are current market standards
> Also 8-core sockets available now and could become a standard in next couple of year• Most common rack servers now have two sockets
> 8-core (or more ) systems are the norm with trend going to 12-16 core systems soon• Most Servers have internal drives > 146 GB
> Denser in capacity, smaller in size but essentially same or lower speed
> More denser in case of SATA-II drives
Current Market Trends in Software
• Software (including Operating Systems) have yet to fully catch up with multi-core systems
> “tar” still single process utility• Horizontal Scaling helps a lot but not a good clean solution for multi-core systems
• Virtualization is the new buzzword for Consolidations
> Hides the fact that the software is not able to fully capitalize the extra cores
• Research being done on new paradigms
> Complexity of parallelized software is huge
Current Market Trends in Data
• 12 years ago, a 20GB data warehouse was considered a big database
• Now everybody talks about 200GB-5TB databases
• Some 2005 Survey numbers:
> Top OLTP DB sizes = 5,973 GB to 23,101 GB
> Top DW DB Sizes = 17,685 GB to 100,386 GB
> Source http://www.wintercorp.com/VLDB/2005_TopTen_Survey/TopTenWinners_2005.asp• Some 2007 Survey numbers:
> Top DB sizes = 20+ TB to 220 TB ( 6+ PB on tape)
> Source http://www.businessintelligencelowdown.com/2007/02/top_10_largest.html
May 2, 2008
OpenBSD 4.3 released!
OpenBSD 4.3 was released on May 1, 2008.
What’s New?
About OpenBSD:
The OpenBSD project produces a FREE, multi-platform 4.4BSD-based UNIX-like operating system. Our efforts emphasize portability, standardization, correctness, proactive security and integrated cryptography. OpenBSD supports binary emulation of most programs from SVR4 (Solaris), FreeBSD, Linux, BSD/OS, SunOS and HP-UX.
OpenBSD is freely available from our FTP sites, and also available in an inexpensive 3-CD set.
April 17, 2008
“The New Alexa Rankings”
http://www.alexa.com/site/company/announcement
My site’s ranking has changed. Was it wrong before?
Your ranking wasn’t wrong before, but it was different. Alexa toolbar users’ interests and surfing habits could differ from those of the general population in a number of ways, and we described some of those possible differences on our website. While the vast majority of sites’ rankings were unaffected by such differences, we’ve worked hard on our new ranking system to adjust for situations in which they could matter.
The new rankings should better reflect the interests and surfing habits of the broader population of Web users.
April 15, 2008
April 9, 2008
[mailist] How to handling large volumes of data on PostgreSQL?
mailing list: pgsql-admin.postgresql.org
from: Johann Spies
..loaded about 4,900,000,000 in one of two tables with 7200684 in the second table in database ‘firewall’, built one index using one date-field (which took a few days) and used that index to copy about 3,800,000,000 of those records from the first to a third table, deleted those copied record from the first table and dropped the third table.
This took about a week on a 2xCPU quadcore server with 8Gb RAM..—
Table paritioning is need.
distribute tables across different disks through tablespaces.Tweak the shared buffers and work_mem settings.
RAID5/6 are very,very slow when it comes to small disk *writes*.
At least a hardware RAID controller with RAID 0 or 10 should be used, with 10krpm or 15krpm drives. SAS preferred.
as on SATA the only quick disks are Western Digital Raptor.
look at a view called pg_stat_activity. Do: select * from pg_stat_activity;
March 25, 2008
关于 JDBC, ORM, OLTP, OLAP…
Non-ORM layers over JDBC?
If you want JDBC access that is more closely integrated into the language I
would suggest using Groovy. It REALLY simplifies JDBC access because of
Groovy’s dynamic typing, which is basically the same thing as using variant
data types in C++, at least syntactically. Groovy’s way of executing JDBC’s
statements is also much easier to use. Groovy compiles to Java class files
and the JVM doesn’t know the difference. The groovy runtime/library is just
a jar file that you stick on your classpath.ORM for me works really well in OLTP situations. If I am doing pure OLTP I
rarely need to go outside of my ORM access layer, which is Hibernate.
Hibernate’s query language (HQL) has lots of features to make writing SQL
queries easier and lots of features to minimize performance problems. If you
are used to SQL, it make take a little getting used to because HQL is more
abstract than SQL. It’s like making a jump from C to Java, more abstraction,
less code, less raw power.If you have lots of screens where users are basically building up sql queries,
using forms, then Hibernate’s query by criteria makes this easy because you
are not longer manually building up SQL (or HQL) queries by hand (which is
really error prone). All of my complicated search screens use this feature
of Hibernate.ORM falls down badly for two things: 1) OLAP style database work and 2) Batch
processing. OLAP depends way too much on specific database facilities to
make things fast, which Hibernate can’t take advantage of. Batch processing
chokes because Hibernate will cache too much because it is trying to optimize
OLTP style interactions.–David Clark
March 20, 2008
TB级 PostgreSQL 拾零
有关TB级以上 PostgreSQL 数据库的一些信息：
Well I can’t speak to MS SQL-Server because all of our clients run
PostgreSQL ;).. I can tell you we have many that are in the 500GB -
1.5TB range.All perform admirably as long as you have the hardware behind it and are
doing correct table structuring (such as table partitioning).Sincerely,
–Joshua D. Drake
We have several TB database in production and it works well on
HP rx1620 dual Itanium2, MSA 20, running Linux. It’s read-only storage for
astronomical catalogs with about 4-billions objects. We have custom
index for spherical coordinates which provide great performance.–Oleg
I had a client that tried to use Ms Sql Server to run a 500Gb+ database.
The database simply colapsed. They switched to Teradata and it is
running good. This database has now 1.5Tb+.Currently I have clients using postgresql huge databases and they are
happy. In one client’s database the biggest table has 237Gb+ (only 1
table!) and postgresql run the database without problem using
partitioning, triggers and rules (using postgresql 8.2.5).–Pablo
I think either would work; both PostgreSQL and MS SQL Server have
success stories out there running VLDBs. It really depends on what you
know and what you have. If you have a lot of experience with Postgres
running on Linux, and not much with SQL Server on Windows, of course the
former would be a better choice for you. You stand a much better chance
working with tools you know.–Pablo Alcaraz
All of those responses have cooked up quite a few topics into one. Large
databases might mean text warehouses, XML message stores, relational
archives and fact-based business data warehouses.The main thing is that TB-sized databases are performance critical. So
it all depends upon your workload really as to how well PostgreSQL, or
another other RDBMS vendor can handle them.Anyway, my reason for replying to this thread is that I’m planning
changes for PostgreSQL 8.4+ that will make allow us to get bigger and
faster databases. If anybody has specific concerns then I’d like to hear
them so I can consider those things in the planning stages.–Simon Riggs
March 5, 2008
Oracle on eBay goes over 5 petabytes
Oracle on eBay goes over 5 petabytes
Curt Monash, the Harvard teen PhD genius., notes that eBay’s Oracle database has now exceeded five petabytes:
From Oliver Ratzesberger’s LinkedIn profile:
Our systems process in excess of 10 billion records per day, serving thousands of users and delivering hundreds of millions of queries per month in a true global 24×7 operation with distributed teams around the globe on systems over 5 PB in size (largest single system >1.4PB).
With multi-petabyte databases becoming common, it’s only a matter of time until Oracle will support Petafiles . . . .
eBay is noted as one of Oracle’s premier web apps, with over 20 billion transactions per day:
* Over 212 million registered users
* Two Petabytes of Data
* 26 billion SQL executions per day
* 99.94% available
* One billion page views per day
* Uses Sun e10k serversCurt Monash also notes:
“eBay’s biggest analytic database is 1.4 petabytes of disk, holding between 1/2 and 1 petabyte of user data, and running (I’m pretty certain) on Teradata.”
February 27, 2008
FreeBSD 7.0 available for download
FreeBSD 7.0 is available for download!~
FreeBSD LogoFreeBSD 7.0 Stable, the long awaited next major release of the popular open source BSD operating has finally arrived! The ISO’s can now be downloaded from the FreeBSD FTP or torrent servers. We still have to wait for the official announcement
Congratulations to the FreeBSD release engineering team for another great release of FreeBSD (release notes).
* What’s new in FreeBSD 7.0
* What is (was) cooking for FreeBSD 7.0— by Gerard
DragonFly-1.12.0 RELEASED
26 February 2008
The DragonFly-1.12.0 Release is ready!
What is DragonFly BSD?
DragonFly is an operating system and environment originally based on FreeBSD. DragonFly branched from FreeBSD in 2003 in order to develop a radically different approach to concurrency, SMP, and most other kernel subsystems.
February 25, 2008
February 23, 2008
支付宝用户规模参考数据
注册用户：5200万
截至2007年9月17日
——————————————————————————
日交易总额：1.9亿元
日交易笔数：100 万笔 (10-20笔/秒)
—————————————
企业级客户：50多万
C类用户：5200万
PostgreSQL 规模参考：400G+容量，N亿条记录
400G容量，N亿条记录
With our previous solution, we found it difficult to support databases over 50GB. Now, because of PostgreSQL, we are easily working with databases with 400GB+ of data and hundreds of millions of records… It has brought us to our markets spotlight and has played a primary role in our continuing growth of revenue.
Joshua Marsh, Listfusion
