..::Planet PostgreSQL::..

When I started Planet PostgreSQL project  about 6.5 years ago, we had only very few (almost none!) bloggers -- but we had to start at some point. As time moved on, lots of people started blogging, and we had invaluable blog posts about various PostgreSQL related topics.

When community decided to fork the planet project because of valid reasons, it increased the visibility of blogs, because it was also feeding postgresql.org main page. It also has more bloggers. However, we had tons of people who were following "my" planet, because they already bookmarked it and never ever heard about new planet.

Anyway, I'll keep it short: As of today, I will be stopping updating this planet. This will be the latest post on this planet. Please use http://planet.postgresql.org  as the new Planet PostgreSQL URL.

I will redirect domains in a few months after making sure that (almost) everyone updated their bookmarks.

There will be no changes in people.planetpostgresql.org.

Regards, Devrim

I’m barely back from a 2-week vacation, and still easing my way back into Portland living.

Looking back on 2010, the highlights were:

• Listed as a major contributor to PostgreSQL. In typical community-style, I found out by chance by looking at the contributors website one day, weeks after the change was made. Sneaky devils.
• Gave talks at LinuxConf.AU in New Zealand, LibrePlanet in Boston, PgCon in Ottawa, CouchCamp in the Bay area, Grace Hopper Celebration of Women in Computing in Atlanta, LinuxFest Northwest in Bellingham, WA, Google Summer of Code Mentor’s Summit in Mountain View, and keynoted DevNation in Portland.
• Helped run the first ever IgniteGov at GOSCON in Portland!
• Spoke at and participated in KiwiFoo and FooCamp. KiwiFoo changed my life. At some point, I’d love to live in New Zealand.
• Made stickers with Reid and distributed OSFY, FSFY and OALFY stickers across the world.
• Lead the second successful Open Source Bridge conference. The second edition brought in Danny O’Brien, as he started his work with the Committee to Protect Journalists, Leigh Honeywell, who chronicled the rise of hackerspaces around the world, and Mayor Sam Adams. Danny’s talk was an inspiration to me, and I’m looking for opportunities to create better and more secure software for people who fight political oppression.
• Started work at Emma, a lovely group of people and tons of interesting problems to work on.
• Learned Python. Well, still learning python.
• Took a couple vacations – Fourth of July weekend in Nashville, TN and two weeks in St Croix in December.
• Spent a week in Montana with my mom for the first time in many years and had Thanksgiving with family at her mountain-man hideout near Tally Lake.
• Drove to and from from Montana. I love road trips.
• Started the PDX11 site to help publicize what the city and citizens are working together to create an even better and more inviting software community in Portland.
• Worked on trolluniversity.com with Duke and Bart. Lots of scotch, ontologies and awkward jokes expected in the future.
• Saw one of the world’s largest tesla coils turned on and run in a giant warehouse. Twice. And learned a tiny bit more about lightning, Tesla and high voltage.
• Connected with a ton of new friends in open source communities, and hope to make many more in the new year.

I’m sure I left a few things out. I wish I would have written this post last year for 2009, as it was also an amazing year, full of travel, new experiences and wonderful people. Maybe I’ll do that and just backdate it for myself. I forgot that I’d done a picture inspiration last year for 2010. I substituted scuba diving for flying – cheaper and quite a bit safer. Otherwise.. mostly all came to pass.

Another thing about this year: I’m still not sure what it is that I want to become. I’ve had many conversations over the last six months with women who are at a similar stage of life – stable work/career, considering having children, loving travel, obsessed with software and communication.

What comforted me is that they were sitting with the ambiguity of it all as well, and finding ways of sifting through probabilities to home in on what’s truly important. I think happiness is an important factor in decision making about life’s goals, but there’s a big part of me that’s more apt to go for interestingness instead. So, while sometimes things have gone off the rails or been incredibly difficult in the last year, all the experiences I listed above have made the sacrifices and contemplative time worth it.

Related posts:

1. Foocamp 2010: lovely, expectant, reflective

On 29th of December, Robert Haas committed interesting patch, which does: Support unlogged tables.   The contents of an unlogged table aren't WAL-logged; thus, they are not available on standby servers and are truncated whenever the database system enters recovery. Indexes on unlogged tables are also unlogged. Unlogged GiST indexes are not currently supported. (edited [...]

Over the weekend, I discovered an open source tool for version control visualization, Gource. I decided to put together a few videos to showcase End Point's involvement in several open source projects.

Here's a quick legend to help understand the videos below:

	The branches and nodes correlate to directories and files, respectively. In the case of the image to the left, the repository has a main directory with several files and three directories. One of the child directories has one file and the other two have multiple files.
	A big dot represents a person, and a flash connecting the person and a file signifies a commit.
	White + blue dots represent current End Point employees.
	White + grey dots represent former End Point employees.
	White dots represent other people, out there!

The Videos

Interchange from endpoint on Vimeo.

pgsi from endpoint on Vimeo.

Spree from endpoint on Vimeo.

Bucardo from endpoint on Vimeo.

One of the articles that references Gource suggests that the videos can be used to visualize and analyze the community involvement of a project (open source or not). One might also be able to qualitatively analyze the stability of project file architecture from a video, but this won't reveal anything definitive about the code stability since external factors can influence file structure. For example, since I am intimately familiar with the progress of Spree, I can identify when Spree transitioned to Rails 3 in the video, which required reorganization of the Spree core functionality (read more about this here and here).

In the case of this article, I wanted to highlight End Point's involvement in a few open source projects where we've had various levels of involvement. We've contributed to Interchange since 2000. We've been involved in Spree less lately, but had more presence in early 2009. In the smaller projects Bucardo and pgsi, End Point employees have worked on a team to be the primary contributors to the projects in addition to a few external contributors. Open source is important to End Point, and it's great to see our presence demonstrated in these cute videos.

PostgreSQL 9.1alpha3 released! Get it here: http://developer.postgresql.org/pgdocs/postgres/release-9-1-alpha.html Here's how to test it: http://wiki.postgresql.org/wiki/HowToBetaTest

Hello
I released new version of pst collection. This version was enhanced about a regress tests and documentation. At the same time I renamed this package to PL toolbox, because functions from this package are intended for using in stored procedures.
Happy New Year

One of my Christmas gifts this year was this charming little statue of Ganesha, the Hindu God with an elephant's head. Some years ago I was going to create a piece of software named for Ganesha, a Postgres-backed blog, but as often happens life and other things got in the way. He's used in the logo for Dunslane Consulting LLC, and is a very appropriate God to be associated with PostgreSQL, being recognized by Jains and Buddhists as well as Hindus (so he has wide appeal), and is a patron of arts and sciences and deva of intellect and wisdom, according to Wikipedia.

And he's cute.

PostgreSQL has a nice functions for operations over bitmapset. A bitmapset is set of positive integers. It should be smaller than array, and, what is important for me. it support very fast test if some value is in set or not. PostgreSQL internally use this functions, but there are not possibility to use it from SQL (PL/pgSQL). I wrote a wrapper for this functionality and moved it to PST collection - http://pgfoundry.org/frs/download.php/2909/pstcoll-10-12-29.tgz.so there is some preview:

pavel=# select pg_column_size(pst.bitmapset '{1,2,3,4}');
 pg_column_size 
----------------
              8
(1 row)

pavel=# select pg_column_size(array[1,2,3,4]);
 pg_column_size 
----------------
             40
(1 row)

pavel=# select pst.add_members('{}', 1, 2, 4,8);
 add_members 
-------------
 {1,2,4,8}
(1 row)

pavel=# select pst.is_member(pst.add_members('{}', 1, 2, 4,8), 8);
 is_member 
-----------
 t
(1 row)

pavel=# select pst.bitmapset_union('{1,2,3}','{6,2,9}');
 bitmapset_union 
-----------------
 {1,2,3,6,9}
(1 row)

Without bitmapsets we have to use a arrays. But bitmaps are better for storing some flags - it's more adequate tool.

pavel=# select count(*) from omega;
  count  
---------
 1010000
(1 row)

pavel=# select bitmapset_collect(a) from omega;
    bitmapset_collect     
--------------------------
 {0,1,2,3,4,5,6,7,8,9,10}
(1 row)

Time: 558.667 ms

pavel=# select array_agg(distinct a) from omega;
        array_agg         
--------------------------
 {0,1,2,3,4,5,6,7,8,9,10}
(1 row)

Time: 3859.567 ms

Using a bitmapset is about 7x faster.

pavel=# select del_members('{2,3,4,5,1}',2,3);
 del_members 
-------------
 {1,4,5}
(1 row)

pavel=# select bitmapset_is_subset('{1,2,3}','{1,3}');
 bitmapset_is_subset 
---------------------
 f
(1 row)

pavel=# select bitmapset_overlap('{1,2,3}','{1,3}');
 bitmapset_overlap 
-------------------
 t
(1 row)

pavel=# select bitmapset_difference('{1,2,3}','{1,3}');
 bitmapset_difference 
----------------------
 {2}
(1 row)

Probably this simple wrapper can be enhanced - some GiST or GIN index can be nice. The bitmapset is limited only on integers. There is not simple way to use it together with enums for example.

In a prior article we did a review of PostgreSQL 9 Admin Cookbook, by Simon Riggs and Hannu Krosing. In this article we'll take a look at the companion book PostgreSQL 9 High Performance by Greg Smith.

Both books are published by Packt Publishing and can be bought directly from Packt Publishing or via Amazon. Packt is currently running a 50% off sale if you buy both books (e-Book version) directly from Packt. In addition Packt offers free shipping for US, UK, Europe and select Asian countries.

For starters: The PostgreSQL 9 High Performance book is a more advanced book than the PostgreSQL 9 Admin Cookbook and is more of a sit-down book. At about 450 pages, it's a bit longer than the PostgreSQL Admin Cookbook. Unlike the PostgreSQL 9 Admin Cookbook, it is more a concepts book and much less of a cookbook. It's not a book you would pick up if you are new to databases and trying to feel your way thru PostgreSQL, however if you feel comfortable with databases in general, not specific to PostgreSQL and are trying to eek out the most performance you can it's a handy book. What surprised me most about this book was how much of it is not specific to PostgreSQL, but in fact hardware considerations that are pertinent to most relational databases. In fact Greg Smith, starts the book off with a fairly shocking statement in the section entitled PostgreSQL or another database? There are certainly situations where other database solutions will perform better. Those are words you will rarely hear from die-hard PostgreSQL users, bent on defending their database of choice against all criticism and framing PostgreSQL as the tool that will solve famine, bring world peace, and cure cancer if only everyone would stop using that other thing and use PostgreSQL instead:). That in my mind, made this book more of a trustworthy reference if you came from some other DBMS, and wanted to know if PostgreSQL could meet your needs comparably or better than what you were using before.

In a nutshell, if I were to contrast and compare the PostgreSQL 9 Admin Cookbook vs. PostgreSQL High Performance, I would say the Cookbook is a much lighter read focused on getting familiar with and getting the most out of the software (PostgreSQL), and PostgreSQL High Perofrmance is focused on getting the most out of your hardware and pushing your hardware to its limits to work with PostgreSQL. There is very little overlap of content between the two and as you take on more sophisticated projects, you'll definitely want both books on your shelf. The PostgreSQL 9 High Perofrmance book isn't going to teach you too much about writing better queries,day to day management, or how to load data etc, but it will tell you how to determine when your database is under stress or your hardware is about to kick the bucket and what is causing that stress. It's definitely a book you want to have if you plan to run large PostgreSQL databases or a high traffic site with PostgreSQL.

PostgreSQL 9 High Performance is roughly about 25% hardware and how to choose the best hardware for your budget, 40% in-depth details about how PostgreSQL works with your hardware and trade-offs made by PostgreSQL developers to get a healthy balance of performance vs. reliability, and another 35% about various useful monitoring tools for PostgreSQL performance and general hardware performance. Its focus is mostly on Linux/Unix, which is not surprising since most production PostgreSQL installs are on Linux/Unix. That said there is some coverage of windows such as FAT32/NTFS discussion and considerations when deploying terabyte size databases on Windows and issues with shared memory on Windows.

Full disclosure: I got a free e-Book copy of this book just as I did with PostgreSQL 9 Admin Cookbook.

Continue reading "PostgreSQL 9 High Performance Book Review"