Community Tech Alliance

Community Tech Alliance (CTA) is a nonprofit data and technology infrastructure platform for progressive change. The company builds and maintains data systems created for the technological needs of progressive organizations. Specifically, CTA leverages existing tools, such as BigQuery, GCP, and Airbyte, and customizes them for data warehousing, integration, and transformation solutions that make storing, visualizing, and sharing data easier for customers.

CTA uses Airplane for several internal tooling needs, such as user and partnership onboarding, permission setting, and data monitoring.

Issues with scaling internal processes

Committed to helping organizations use data effectively, CTA strives to move quickly and be highly responsive to customer needs. After growing its engineering team to 5x the size and its partnership base to 30+ partners in one year, CTA faced challenges managing its partnership infrastructure and internal processes in one place.

The team utilized Terraform at the beginning to help with state management, developing their partnership infrastructure, and global partner management. They quickly ran into challenges using Terraform. Maintaining a single ecosystem for all partners was difficult and Terraform wasn't a sustainable option to scale rapidly since adding new partner configurations manually took time and was error-prone.

The CTA team also relied on a Confluence page to navigate internal operations like provisioning partners. This page contained a long, cluttered list of instructions, scripts, links, and more. This laundry list of items made it difficult for the team to find the right information at the right time and to audit their internal processes effectively.

As CTA's customer base grew, the team realized that Terraform had its limitations and that they needed a better way to manage all their internal processes. They set out to find a more effective way to manage their internal tooling and access operations quickly without compromising speed and security.

Airplane as the solution

CTA's Chief Technology Officer, Michael Fisher, first found Airplane when looking for a scalable solution to Terraform that could also help with their numerous customer support operations. After trying Airplane out for the first time, they were drawn to how quickly they could spin up automation tasks. As they used Airplane more, CTA realized that the platform provided a number of other benefits such as:

• Streamlining manual processes: Building tasks in Airplane helped streamline tedious manual processes. For example, engineers had to manually update partner information individually. Using Airplane, engineers could make one change using a few lines of code and easily bulk-update all partner information at once.
• Supporting fine-grained permissions: Airplane made it easy to set permissions based on different roles within an organization. In Airplane, there are four roles that any user or group can be assigned to: viewers, requestors, executors, and admins. CTA wanted to limit the ability to approve tasks to certain members of the partnerships team. Configuring this in Airplane allowed these users to sign off on new user creation requests, streamlining the process while making it more secure.
• Providing robust audit logging: Everything in Airplane is automatically audit logged. Engineers could easily view run histories and edit tasks without compromising safety. This also made it easy to debug issues and quickly identify when changes were made to customer accounts.
• Centralizing multi-step functions: CTA was able to spin up combined, multi-step functions from various sources, such as Python, Bash, and APIs, easily using Airplane. For example, CTA wanted to automate 1Password access for new users. To do so, they created a multi-step function in Airplane using Python and the 1Password CLI that added a new user to 1Password each time it was executed. This function was then available in Airplane for the team to run within seconds.
• Making operations easy to scale: Airplane allowed CTA to easily add 30+ partners to the platform and ensured that infrastructure, permissions, and process needs were met reliably.

Overall, Airplane has helped CTA engineers save dozens of hours each week that were previously spent manually running tasks and pulling engineers into customer-related requests. Using Airplane, CTA was able to remove engineering bottlenecks in customer support and help resolve customer issues more quickly. Airplane has also made it easy to migrate one-off scripts and other customer-facing tasks into the Airplane ecosystem, serving as a source of truth for internal operations.

Use cases

There are several use cases that CTA uses Airplane for:

• Creating and updating partner accounts: CTA often needs to add new partner accounts or update existing accounts in bulk. Using Airplane, they can easily add or update partner information (e.g., name, email, location) on a one-off basis. If a bulk update is needed for each partner account, CTA uses their "Bulk Update Partners" task to update several partner accounts in seconds.
• Integrating new apps: CTA also uses Airplane to build multi-step processes that require integrations with other applications. For example, they created a multi-step task in Airplane that grants new users 1Password access using Python and the 1Password CLI. An engineer can run this task in Airplane to automate 1Password access for a new user, eliminating the need to manually add new users.
• Monitoring data effectively: Airplane Tasks are a great solution for a number of CTA's data operations. One example is monitoring and cleaning up data tables. CTA runs Airbyte for their extract, transform, and load data syncs. When this process is interrupted, it leaves junk tables in the BigQuery system. CTA uses Airplane to monitor these data tables and remove the junk, ensuring that data tables are always accurate and up-to-date.
• Executing admin operations: Using Airplane, CTA created an interactive dashboard that contains detailed user information, such as activity, user status, and access level, for partners to use. They are able to grant partners the ability to edit this information without giving them access to the underlying database.
• Configuring approval flows for partner operations: Airplane gives users the ability to set granular, role-based access controls and configure approval flows on sensitive operations. For example, CTA created a group for managers of partner operations and granted them the ability to view data, add new users, and make edits to datasets. Non-managers who don't belong in the group must request these changes and cannot execute them directly.

Today, CTA executes 3,000 weekly runs in Airplane across 80+ different tasks. The team finds Airplane easy to use and is building their full suite of customer and data operations into Airplane. The biggest benefit has been the time Airplane saves engineers and support: "Airplane allows us to spend our limited time prioritizing the things that directly impact our clients. We're also able to handle customer requests faster than ever before."