Home Affairs Integration Architecture Strategy

GoSource designed a domain-based integration strategy to reduce complexity across the Home Affairs portfolio and enable more autonomous service delivery.

Challenge

The Home Affairs portfolio is a very large organisation encompassing immigration, customs, policing, and intelligence functions. The existing system integration framework presented severe challenges: over 1,000 distinct integrations between 200+ IT systems used by 30,000+ users, tight coupling between systems imposing complex dependencies that severely limited agility, and database change-data-capture processes that imposed complex “deciphering” ETL work on consumer systems. The portfolio’s 2020 technology strategy demanded both autonomy/agility for individual business domains and high-quality centralised intelligence information for border risk decision-making — a new integration strategy was identified as the key to achieving both goals.

Solution

The previous Home Affairs integration model was based on a centralised ESB-centric service design and implementation function, where most integrations were ad-hoc and project-driven, resulting in a patchwork of tightly coupled systems with little reuse. With system integration previously considered a purely technical activity, the biggest challenge was driving a change in thinking across a large number of IT system teams and their executives — shifting from “how shall we connect A to B for this project?” to “what does our business unit do and how should we expose our services for self-service consumption by any other business unit?”

GoSource’s approach centred on:

• Top-down domain architecture: Developed a portfolio-wide domain and resource architecture defining the required microservice granularity across 35 business domains
• Decentralised delivery: Each domain was empowered to design and implement their RESTful microservices and events within the centrally governed high-level domain architecture
• Executive engagement: Developed a compelling high-level “why-what-how” slide deck to drive executive support, leading to senior approval and capital funding
• Real-world prototyping: Used real examples and “show the thing” prototypes to successfully engage business domains and IT system owners
• Capability building: Implemented a capability building programme to empower IT teams with the skills needed for the decentralised microservice delivery model

Outcomes

• Dramatic complexity reduction: 700 ESB transformations reduced to 16 resources in 5 domains
• Autonomous domains: 30 autonomous business domains linked by simple APIs and events
• Integration architecture documented: Detailed documentation plus a compelling one-page poster now appearing on walls throughout the portfolio
• Executive approval and funding: Architecture approved by executive and embedded in capital plan
• Business domain acceptance: Architecture accepted by business domains and IT system owners through real examples and prototypes
• Technology platforms designed and funded: Shared technology platforms including API gateway and enterprise event hub designed with successful early prototypes
• Capability building programme: Implemented to empower decentralised IT teams for microservice design and implementation
• Transformation plan: Comprehensive plan for 200+ systems and services; engaged and sold ideas to 20+ business areas

Technologies & Methods

• Domain-driven design (DDD)
• RESTful microservice API architecture
• Event-driven architecture (enterprise event hub)
• AWS API Gateway (prototype)
• Apache Kafka (prototype event bus)
• OpenAPI specification
• OAuth 2.0 (authentication)
• Scaled Agile principles
• Capability building and organisational change management

Team Size

3 (one senior architect plus a modeller and API developer)