Intro To Eda
The Enterprise Event Bus is an asynchronous event processing system that spans systems and lines of business at VA. Event-driven architecture uses events — types of actions, such as a Veteran creating a medical appointment or updating their beneficiaries — to communicate with systems that are subscribed to the stream of events. The systems that are producing the events, also known as producers, are decoupled from the systems that are consuming their events, also known as consumers.
The following conceptual diagram illustrates how the different components of the Enterprise Event Bus might come together to produce streams of events. View the full-sized diagram (must be part of VA GitHub organization to view).
We define enterprise events as those that capture significant occurrences within VA, such as when a Veteran’s benefit claim reaches certain meaningful milestones.
We strive to enable connecting previously disparate systems that live across different business lines within VA, such as the Veteran Benefit Administration (VBA) and the Veteran Health Administration (VHA). As such, we are not limited to a specific business line or domain within VA.
It should be noted that Enterprise Event Bus does not aspire to be the only event streaming platform within VA, but one that is available to any team that wants to produce or consume enterprise events. This would allow teams who don’t have the time or resources to set up their own event streaming platform to take advantage of event-driven technologies. It is not intended to subsume or replace any existing VA API ecosystems, nor is it a one-stop shop or mandatory runtime environment for all VA data services; instead, it is intended to enable event-based architecture for current and future integrations between systems.
We often refer to the Enterprise Event Bus team as an opinionated conduit between event consumers and producers across the VA ecosystem. That’s because our research, hands-on experience, and deep knowledge of Apache Kafka best practices have led us to definite opinions regarding:
- the type of events that best fit an Enterprise Event Bus architecture (more on that below)
- the choice of technologies with which we implement the Enterprise Event Bus across different contexts of use
- and the most suitable infrastructure for implementing it.
These enable us to provide producers and consumers with clear guidance to and assist with onboarding and learning about the underlying technologies.
The focus of the Enterprise Event Bus is enterprise events rather than data events. As described above, an event is a specific action that involves a change in data. An enterprise event is a business event that is of potential interest to a broad scope of consuming systems.
For example, when Veterans apply for benefits, they are issued a benefits decision letter detailing VA’s decision about what benefits they are eligible for. The benefits decision letter becoming available is an enterprise event. The specific changes in the underlying data that indicate this document has become available are data events. Teams producing events onto the Enterprise Event Bus are encouraged to frame their events as enterprise events; that is, broadly speaking, an event that describes a concrete business event has the potential to be more useful to a broader audience than specific data change events.
When determining what information an event should contain, a guiding principle is that consumers need events to at least contain enough information to know whether the event is of interest to them, while minimizing the amount of sensitive data in the event payload. In general it is easier to add new pieces of information to a schema than it is to later remove it; thus the system should trend towards leaner events.
Enterprise Event Bus, like most other event-based systems in VA, is based on Apache Kafka, an open-source distributed event streaming platform. It is considered the industry standard for handling real-time data feeds, and is capable of handling the kinds of large scale, high-throughput loads we would expect an enterprise-wide event bus in VA to be able to handle. In addition to being a proven technology, Kafka is already used in a production capacity by other teams in VA, such as the Benefits Integration Platform.
The Enterprise Event Bus uses AWS Managed Streams for Kafka (MSK), a hosted version of Kafka that runs in the VA Enterprise Cloud.
The following diagram illustrates how events pass through the different components of the Enterprise Event Bus system. View the full-sized diagram (must be part of VA GitHub organization to view).
- Find definitions for acronyms and event-related terms on the Terminology page.
- Watch this brief video for a quick overview of event-driven architecture.
- Learn about event streaming patterns on the Confluent Developer website.
- Watch a presentation about “Lessons Learned and Plans for the Future” by Robert Ezekiel from Booz Allen Hamilton.
- Watch a presentation about “Improving Veteran Benefit Services Through Efficient Data Streaming” by Robert Ezekiel from Booz Allen Hamilton
- The Enterprise Event Bus Team offers consultations and is happy to answer questions. Reach out to us.