PGO 4.7, the Postgres Operator: PVC Resizing, GCS Backups, and More

We're excited to announce the new version of PGO, the open source Postgres Operator from Crunchy Data version 4.7! There's a lot of really cool features that make it easy to deploy production Postgres clusters on Kubernetes. In this release, we focused on adding enhancements around "Day 2" operations (e.g. PVC resizing), allowing for backups to be stored in Google Cloud Storage (GCS), improvements to monitoring, and included more Postgres extensions!

Let's take a quick tour of all of the new features in PGO 4.7 and see them in action.

What's New in PGO 4.7 At A Glance

• Resize Persistent Volume Claims (PVCs)
• Store Backups in Google Cloud Storage (GCS)
• Restore / Clone a Postgres Cluster Across Namespaces
• Monitoring Improvements: Query Performance, Backups, Alerts Dashboard
• Toggle TLS in Active Postgres Clusters
• Extensions: PostGIS 3.1, pg_partman, pg_cron, TimescaleDB

Resize Persistent Volume Claims (PVCs)

There are many things that can cause someone to need more storage in their database system, such as an increase in a backup retention policy or organically through data growth. In Kubernetes, this means increasing the size of a persistent volume claim, or PVC.

PGO 4.7 lets you increase the PVC size for your Postgres clusters in a non disruptive way! You can do this by editing the size attribute on the various storage options on the pgclusters.crunchydata.com or pgreplicas.crunchydata.com custom resources. You can also use the pgo update cluster command to resize the Postgres cluster, pgBackRest repo, and WAL PVC sizes with the --pvc-size, --pgbackrest-pvc-size, and --wal-pvc-size flags respectively.

When the PVC size is increased, PGO performs a rolling update to minimize (or eliminate!) any downtime that could occur. The bonus is that the PVC resize method is backwards compatible down to Kubernetes 1.11 / OpenShift 3.11.

PGO 4.7 lets you resize the following PVCs:

• The Postgres data PVC, i.e. the PVC that holds your database
  • The PVC of an individual instance in the PostgreSQL cluster
• The pgBackRest repository PVC
• The optional WAL directory PVC
• The pgAdmin 4 PVC

Store Backups in Google Cloud Storage (GCS)

Google Cloud Storage (GCS) is a blob storage service available in the Google Cloud Platform. pgBackRest 2.33 added native support for backup management using GCS. And now, PGO, natively supports storing backups in GCS!

This release brings equivalent functionality between PGO's support for S3 and GCS. This includes support for multi-Kubernetes cluster deployments!

Specifically, PGO 4.7 adds the following attributes to the pgclusters.crunchydata.com custom resource for configuring backup storage with GCS:

• BackrestGCSBucket (required)
• BackrestGCSEndpoint
• BackrestGCSKeyType

The pgBackRest repo Secret supports a key called gcs-key that references a GCS credential. For more information on setting up a Postgres cluster with backups stored in GCS using custom resources, please see the custom resources section of the documentation.

Similarly, additional flags are now available in the pgo create cluster command to enable GCS support, including:

• --pgbackrest-gcs-bucket
• --pgbackrest-gcs-endpoint
• --pgbackrest-gcs-key -- this should reference a file in your local environment. Don't worry; the pgo client can handle uploading it!
• --pgbackrest-gcs-key-type

For more information, please refer to the documentation.

Restore / Clone a Cluster Across Namespaces

PGO has long supported managing Postgres clusters across different Kubernetes Namespaces. PGO 4.7 takes that a step further and lets you restore or clone Postgres clusters across Kubernetes namespaces! This is helpful when, say, you need to copy data between a production and test instance.

For example, to create a cluster called hippo from data in a cluster called elephant in namespace production, you can execute the following command:

pgo create cluster hippo --restore-from=elephant --restore-from-namespace=production

and PGO will create hippo from elephant. This also works by setting the namespace attribute in the restoreFrom block in the pgclusters.crunchydata.com custom resource.

Monitoring Improvements: Query Performance, Backups, Alerts Dashboard

Monitoring your Postgres clusters in production is important. Very important.

PGO 4.7 makes several enhancements to monitoring Postgres databases in Kubernetes, including:

• Query Performance Insights: analyze the "top N" views (longest running queries, average runtime, total queries run) to better understand where to troubleshoot your SQL.
• More information about the state of your backups, including recovery window, any missing WAL archives, backup size, and more.
• A new "alerting dashboard" that provides summary information about system health and alert history.

There are also general improvements to the overall user experience of the monitoring visualizations to make it easier to determine the health of your Postgres cluster!

Toggle TLS in Active PostgreSQL Clusters

You developed your application on Postgres and now it's time to move it to production settings. But wait, you forgot to add TLS!

Fortunately, PGO now allows you to enable or disable TLS in an active Postgres cluster. This has the added benefit of being able to point a Postgres cluster at different Secrets, e.g. as part of certificate rotation.

The pgclusters.crunchydata.com custom resource now allows for the following attributes in its specification to be edited:

• tls.caSecret
• tls.replicationTLSSecret
• tls.tlsSecret
• tlsOnly

Additionally, there are a bunch of new flags for the pgo update cluster command that support updating TLS settings:

• --disable-server-tls: removes TLS from a cluster
• --disable-tls-only: removes the TLS-only requirement from a cluster
• --enable-tls-only: adds the TLS-only requirement to a cluster
• --server-ca-secret: combined with --server-tls-secret, enables TLS in a cluster
• --server-tls-secret: combined with --server-ca-secret, enables TLS in a cluster
• --replication-tls-secret: enables certificate-based auth between Postgres instances.

Extensions: PostGIS 3.1, pg_partman, pg_cron, TimescaleDB

PGO 4.7 updates PostGIS to version 3.1 and brings in support for pg_partman, pg_cron, TimescaleDB. We'll have a few blog posts coming up on how to use these extensions in your Kubernetes Postgres clusters!

But Wait, There's More!

There are a lot more features in PGO 4.7, you can read all about them in the full release notes:

You can get started with PGO 4.7 in your Kubernetes environment by running the two commands below:

kubectl create namespace pgo
kubectl apply -f https://raw.githubusercontent.com/CrunchyData/postgres-operator/v4.7.0/installers/kubectl/postgres-operator.yml

or by following the quickstart.

Feedback Welcome

We're excited for this latest release of PGO, the open source Postgres Operator from Crunchy Data and helping you manage your Kubernetes native Postgres. If you give PGO 4.7 a test, let us know how it's going on Twitter or LinkedIn. And if you’re already using our Operator, consider starring it on GitHub.