John McBride

We recently released integrations with the OpenSSF Scorecard on the OpenSauced platform. The OpenSSF Scorecard is a powerful Go command line interface that anyone can use to begin understanding the security of their projects and dependencies. It runs several checks for dangerous workflows, CICD best practices, if the project is still maintained, and much more. This enables software builders and consumers to understand their security posture, deduce if a project is safe to use, and where improvements to security practices need to be made.

In September of 2022, the European Parliament introduced the “Cyber Resilience Act”, commonly called the CRA: a new piece of legislation that requires anyone providing digital products in the EU to meet certain security and compliance requirements.

As data intensive backend applications scale and grow, with larger data sets scaled out to higher availability, performance bottlenecks can quickly become major hurdles. Processing requests that once took mere milliseconds can suddenly become multi-minute problems.

In this blog post, let’s take a look at some recent optimization strategies the OpenSauced pizza micro-service recently underwent. This backend service is a Go server that processes git commits by request, sometimes processing thousands of commits in one single request. You can almost think of it as a real time batch processor that can be called by arbitrary clients to fetch and process git commits within an agnostic git repo.

These commits eventually are all indexed within a Postgres database. Most of these optimizations revolve around “batching” the Postgres calls instead of going one by one. For simplicity in our examples, we’ll be using an arbitrary table called “my_table” with data that fits into the “my_data” column. Let’s dive in and take a look at how we can optimize!

Over the last few weeks, the OpenSauced engineering team has been building a service we're calling the “pizza oven.” This service indexes commits within bespoke git repositories and can be used to generate insights based on those commits. This all gives us the ability to create interesting metrics around open source project velocity, “time to merge”, the who’s who of contributors, and more; all by indexing and parsing the git commits! We’ve been experimenting with many different models and have created an interesting solution for increased performance and availability of the service.