Performance has been steadily deteriorating as you use your Postgres system.
Use the VACUUM ANALYZE command in psql or the vacuumdb command-line tool.
When you delete records from Postgres it doesn't reclaim the space, it just marks them deleted and gets on with other business. From psql you can issue a VACUUM command to reclaim space from a table.
Its second function is to update index statistics. Your indexes will always be accurate without this, but Postgres uses the statistics to know when and where to use an index rather than scanning the table.
If you want to see what is happening you can add the VERBOSE qualifier. So:
VACUUM VERBOSE ANALYZE diary;
Note - those of us who speak the Queen's English can use ANALYSE instead in recent versions of Postgres.
[NOTE - blocks updates but not reads - check this]
The vacuumdb command works similarly, but can process a whole database in one go. See the man page for information.