Event Driven Architecture

Event-Driven Architecture (EDA) is a software architecture paradigm promoting the production, detection, consumption of, and reaction to events. An event can be defined as a significant change in state, and this architecture revolves around the idea of responding to these actions or changes. EDA is widely used because it allows for high responsiveness, flexibility, and scalability in applications.

1. Fire and Forget: User activity data sent to a logging service, no response required.
2. Reliable Delivery: E-commerce applications retry until orders are processed.
3. Infinite Stream of Events: Twitter processes endless stream of tweets.
4. Anomaly Detection/Pattern Recognition: Real-time fraud detection in credit card transactions.
5. Broadcasting: Stock price changes broadcasted to subscribed traders.
6. Buffering: Netflix loads video chunks for uninterrupted playback.

 

	Description	Example
Event Streaming	This pattern involves a continuous, real-time flow of events that are produced and immediately processed. There is no explicit subscription needed as all events are available to be processed	A music streaming app like Spotify uses event streaming. When a user plays a song, the data (song) is streamed or sent continuously allowing the user to listen to it in real time
Publisher/Subscriber	This pattern involves publishers producing events and subscribers actively choosing which events they want to listen to and react to. It allows for a decoupling of the event source from its consumers	A blog website can use this pattern. When a blogger (publisher) posts a new blog, only the users (subscribers) who have subscribed to that particular blogger or topic will be notified.

 

Saga Design Pattern

• The Saga design pattern is used to manage transactions that span multiple services in a microservices architecture.
• It maintains data consistency across services with a series of local transactions.
• Example: In an e-commerce app, placing an order may involve multiple services such as inventory, payment, and shipping. The Saga pattern ensures that either all these operations succeed, or, if one fails, appropriate compensating transactions are executed.

  

	Orchestration	Message Broker
Description	Orchestration involves a central controller (orchestrator) that dictates how the services will interact with one another.	A message broker pattern uses a communication platform for services to send messages between each other.
Key Point	Provides centralized control and is simple to understand, but can create a bottleneck.	Decouples services and allows asynchronous communication, but adds complexity and requires careful handling of message delivery.

Command Query Responsibility Segregation (CQRS)

This pattern separates read and write operations for improved performance and scalability.

Use Case 1: Separate Read and Write

Separate Read and Write in an ecommerce system, frequent read operations like browsing products can be optimized separately from less frequent write operations like inventory updates. 

Use Case 2: Materialized View

Materialized View, combining data from multiple microservice to create a joined view

 

Event Sourcing

• Event Sourcing is a design pattern that ensures all changes to application state are stored as a sequence of events. These events can be queried, and the state can be reconstructed at any point of time.
• It's useful in systems where complete history of actions is necessary, like in banking transactions where all transactions are recorded and can be used to trace the current balance.

Contract Testing

• Contract testing verifies microservices interaction using expected requests and responses.
• It makes issue detection easier without full integration.
• It allows independent development and scaling of microservices
• Pact is a tool that can be used for contract testing

Domain Driven Design

 

Architectural Patterns  :  Reusable Solution Pattern

1. Communication Patterns

   - SOA: Services provided through a protocol, like separate services in banking systems.

   - Message Bus Architecture: Components communicate through a common channel, e.g., services in an e-commerce platform. 

2. Structural Patterns

   - Layered Architecture: Separates concerns into layers, like distinct layers in a mobile app.

   - OOAD: Uses visualization and abstraction for system models, e.g., modeling in video game design. 

3. Deployment Patterns

   - Client-Server: Server provides services to clients, like a web server serving browsers.

   - 3-Tier Architecture: Separates application into data, logic, and presentation tiers, e.g., an e-commerce site. 

4. Data Patterns

   - Data-Centric Design: Focuses on data and its transformation, like in a data analytics system.

   - DFD: Graphical representation of data flow, e.g., visualizing data flow in a retail management system. 

5. Domain Patterns

   - Domain-Driven Design: Focuses on the core domain and logic, such as patient care in a healthcare system.

Domain-Driven Design : Focus is on the business domain

• Supporting : Auxiliary functionality, like an authentication module in software applications.

• Core : Main business logic and functionalities, separating factor on the business like product catalogue

• Generic : Necessary but not business-specific functionalities, like a logging module in a system.

Bounded Context

In Domain-Driven Design, a Bounded Context is like a fenced garden where specific models exist. Imagine a university with departments like Administration, Academics, and Sports. Each department, with its own rules and processes, represents a Bounded Context. The term "registration" might mean course enrolment in Academics, but signifies sports event sign-up in Sports.

In software, Bounded Contexts maintain model integrity within each area, reducing confusion and enhancing system design clarity.

Strategic Pattern

 

• Big Ball of Mud: A system with no clear structure, like a patched, extended legacy system.

• Anti-Corruption Layers: Protects your system from poorly designed components, acting as a translator, e.g., when integrating with an old billing system

• Separate Ways: Developing independent systems when shared models aren't beneficial, like separate systems for production and HR in a company.

• Open Host Services: Systems expose their functionality in a technology-agnostic manner, e.g., a bookstore's inventory system exposing data through a RESTful API.

• Conformist: Conforming to an existing model when maintaining a separate one isn't practical, e.g., a new microservice adopting the data models of an existing system.

• Customer/Supplier Team: One team acts as a customer and another as a supplier, like the backend team providing APIs and the frontend team using them.

• Shared Kernel: A common model shared between different systems, like two microservices sharing a product catalogue model.

Tactical Pattern

·  Entity, Value, Aggregate Object: Entities are unique, values are immutable, and aggregates are associated objects treated as a unit. In an e-commerce app, a User is an entity, their Address is a value, and a User with their Orders is an aggregate. 

·   Repository Pattern: This pattern mediates between the domain and data mapping layers like an in-memory object collection. In a book store app, a repository handles data operations for books. 

·  Domain, Application, Infrastructure Services: Domain services encapsulate business logic; application services delegate work; infrastructure services communicate with external systems. In a banking app, interest calculation is a domain service, money transfer is an application service, and email notification is an infrastructure service.

·   Anemic and Rich Model: Anemic model separates data and behavior, while a rich model encapsulates them together. In an anemic model, an Order object holds data and a separate service handles logic. In a rich model, the Order object contains data and methods to manipulate it.

Sizing a planned deployment

 

Software Architecture for a Scalable System

 

Strategy for a Simple Scalable Architecture.....

1. Vertical Scaling your computer. User faster computer

2. Add load balancer between layer.

3. Add CDN for the static content.

4. Scale Database by adding read replica

5. Cache expensive operation and complex query result

6. Use microservice to enable flexible scaling

7. Use message system

8. Use non-relation database like Cassandra whenever possible

Now, Large scale scaling 

Leveraging AI for Enhanced Web and API Security - OWASP

The Security Dimension 

1. OWASP Top 10: A common framework for identifying and mitigating web application security risks.

2. Authentication & Authorization: Verifying user identity and controlling access to resources.

3. Secure Communication: Encrypting data in transit and at rest to prevent eavesdropping.

4. DDoS Mitigation: Protecting against Distributed Denial-of-Service attacks that overwhelm systems.

Security Strategy

1. Shift Left Approach: Integrating security earlier in the development lifecycle.

2. Secure Coding Practices: Writing code that is resistant to vulnerabilities.

3. CI/CD Pipeline Integration: Automating security checks throughout the development process.

Security Shift Left with Secure Coding Practices

OWASP Issue evaluation in  Application Layer(L7)

Security Rules Integrated with Developer IDE

Pre-Commit Stage Gate

Advantages:

Identifying security issues early in development

Enforcing secure best practices

Less vulnerable code in shared repository

Explore:

Availability of security plugins for developer IDEs

Synk, Checkmarx one with VSCode

OWASP IDEVulScanner – integration available with VSCode and intellij

AI-powered Security in the CI Pipeline

AI-enabled Security Tools: Leverage AI algorithms to identify vulnerabilities during the pre-merge stage of the CI pipeline.

Pre Merge Stage Gate

Benefits:

Proactive identification of vulnerabilities.

Faster remediation process.

Less vulnerable code in the codebase master

AI-powered Security tools that can be integrated in the CI Pipeline

AI-powered Security in the CD Pipeline

Pre-deployment Stage Gate: Analyse security posture before deployment.

Security Sandbox: Simulate real-world attacks in a controlled environment.

OWASP L4 Evaluation: Assess vulnerabilities related to network security.

GenAI for Analysis: Leverage AI to analyze security data from the sandbox environment.

Go/No-Go Decision: Make informed deployment decisions based on security assessment results.

AI enabled CD Security Scan tool

Monitoring and logging

Post-Deployment Stage

Continuous automated Monitoring and logging

User GenAI tool to analyze and report any security threat

AI-enabled security tools that can help analyze and report security threats - Splunk, IBM QRadar,Securonix,Rapid7 InsightIDR, Exabeam

AI enabled monitoring tool

Performance Architecture

 

Let's Begin....

Performance Architecture is a strategic enabler that contributes significantly to the success of an organization, ensuring high-quality software systems that drive user satisfaction and business growth.

Performance Metrics 

• Response Time: 

Importance: Crucial for a smooth user experience and user satisfaction.

Use Case: In e-commerce applications, product details should load promptly after a user click.

Summary: Minimizing response time enhances user experience and prevents application abandonment.

• Throughput:

Importance: Measures a system's efficiency in handling high volume transactions.

Use Case: A banking system needs to manage high transactions smoothly during peak hours.

Summary: Optimizing throughput ensures system stability during high-load periods, improving customer satisfaction.

Some consideration 

•  Response Time Percentile Distribution: It's a method that shows the variation in response times. For instance, the 95th percentile time is the time that 95% of responses are faster than. This helps identify outliers in system performance.

• Tail Latency: This refers to the delay of the slowest requests. 

• Performance Degradation: In context of increasing resources, performance degradation refers to how adding more resources might not always lead to improved performance. It could be due to inefficient resource utilization or bottlenecks in the system. Monitoring this helps maintain a reliable user experience.

Performance Enhancement 

Caching: Caching can significantly speed up data retrieval by storing frequently requested data in memory. For instance, an e-commerce website can cache product details to quickly display them to users.

CDN: A Content Delivery Network (CDN) can improve performance by hosting static content closer to users, reducing the load time. For example, a media streaming site might use a CDN to provide faster video streaming.

Efficient Network Design: An efficient network design reduces data travel time, enhancing performance. For example, placing servers geographically close to users can reduce latency.

API Gateway: An API Gateway can manage and route requests efficiently, reducing the load on individual services. For instance, it can direct traffic to less-busy servers, improving response times.

Optimizing Data Transfers: Reducing the size of data transferred, like compressing images before sending, can speed up data transfers and improve performance.

Queuing System: A queuing system can handle high volumes of requests without overloading the system. For example, a ticket booking website might use a queue to manage high demand during peak times.

Load Balancer: Imagine a popular ice cream shop with only one server - chaos, right? A load balancer is like hiring more servers for the shop, distributing customers evenly to prevent any server from getting overwhelmed.

• Vertical Scaling: Picture a weightlifter adding more and more weights to his barbell - that's vertical scaling for you. It's about beefing up your existing server with more power - adding more CPU, RAM, or storage. But remember, there's a limit to how many weights our weightlifter can lift!

• Horizontal Scaling: On the other hand, horizontal scaling is like a relay race, where you add more runners (servers) to your team. It effectively distributes the load, and the best part? There's virtually no limit to how many runners you can add. But coordinating them all? That's the real trick!

Performance Enhancement - with Scaling

Data Replication: Think of this as having photocopies of your favorite book in multiple libraries. If one library is too crowded, you can simply go to another to get the book. Similarly, data replication enables quick data access from different servers, improving scalability. 

Data Partitioning: It's like dividing a huge pizza into slices to share among friends. Each slice is easier to handle, right? Similarly, partitioning divides data into manageable chunks, enhancing scalability by distributing the load.

Database Sharding: Imagine splitting a deck of cards among players - that's sharding. It's like data partitioning but on steroids, dividing data across different databases to supercharge your application's scaling ability.

Microservice Architecture: Think of a bustling city divided into specialized districts - one for shopping, one for food, etc. Similarly, a microservice architecture breaks an application into small, specialized services, each scaling independently, just like our city districts!

Performance Enhancement - with Availability

Uptime, Downtime: Uptime refers to the period when an application is available and running. Downtime is when it's not. Maximizing uptime ensures the application is always accessible to users, improving performance. For example, regular maintenance helps increase uptime and reduce unexpected downtime.

MTBF, MTTR: MTBF (Mean Time Between Failures) and MTTR (Mean Time To Repair) are measures of system reliability. A higher MTBF and lower MTTR indicate a more reliable system. For instance, regular system checks can increase MTBF, while having a dedicated support team can reduce MTTR.

Fault Tolerance- Prevention, Detection, Recovery: Fault tolerance is about ensuring the system continues to perform even when issues occur. Prevention, detection, and recovery are key stages. For example, using redundant systems can prevent failures, regular monitoring can help detect issues early, and having a robust recovery plan ensures quick restoration post-failure.

Service Level Agreements (SLAs), Service Level Objectives (SLOs), and Service Level Indicators (SLIs) play a vital role in ensuring the performance of a system:

SLA (Service Level Agreement): This is a contract that defines the level of service expected from a service provider. For example, an internet service provider might promise 99% uptime in their SLA. Regularly meeting SLA helps maintain the performance and reliability of the system.

SLO (Service Level Objective): These are specific measurable characteristics of the SLA such as availability, throughput, response time, etc. For instance, an SLO might be a page load time of under 3 seconds. Setting and achieving SLOs can directly improve the performance of the system.

SLI (Service Level Indicator): These are the metrics used to measure the level of service. For example, the error rate could be an SLI. Monitoring SLIs allows you to spot and fix issues before they affect performance.

Database Essentials

 

Data Model

Database Scaling Strategy

Database Resilience