Construct information pipelines with dbt in Amazon Redshift utilizing Amazon MWAA and Cosmos

Efficient collaboration and scalability are important for constructing environment friendly information pipelines. Nonetheless, information modeling groups usually face challenges with complicated extract, remodel, and cargo (ETL) instruments, requiring programming experience and a deep understanding of infrastructure. This complexity can result in operational inefficiencies and challenges in sustaining information high quality at scale.

dbt addresses these challenges by offering an easier strategy the place information groups can construct sturdy information fashions utilizing SQL, a language they’re already acquainted with. When built-in with fashionable improvement practices, dbt tasks can use model management for collaboration, incorporate testing for information high quality, and make the most of reusable elements by way of macros. dbt additionally routinely manages dependencies, ensuring information transformations execute within the appropriate sequence.

On this submit, we discover a streamlined, configuration-driven strategy to orchestrate dbt Core jobs utilizing Amazon Managed Workflows for Apache Airflow (Amazon MWAA) and Cosmos, an open supply package deal. These jobs run transformations on Amazon Redshift, a totally managed information warehouse that allows quick, scalable analytics utilizing normal SQL. With this setup, groups can collaborate successfully whereas sustaining information high quality, operational effectivity, and observability. Key steps coated embrace:

Making a pattern dbt mission
Enabling auditing inside the dbt mission to seize runtime metrics for every mannequin
Making a GitHub Actions workflow to automate deployments
Organising Amazon Easy Notification Service (Amazon SNS) to proactively alert on failures

These enhancements allow model-level auditing, automated deployments, and real-time failure alerts. By the tip of this submit, you should have a sensible and scalable framework for working dbt Core jobs with Cosmos on Amazon MWAA, so your workforce can ship dependable information workflows quicker.

Resolution overview

The next diagram illustrates the answer structure.

The workflow accommodates the next steps:

Analytics engineers handle their dbt mission of their model management instrument. On this submit, we use GitHub for example.
We configure an Apache Airflow Directed Acyclic Graph (DAG) to make use of the Cosmos library to create an Airflow process group that accommodates all of the dbt fashions as a part of the dbt mission.
We use a GitHub Actions workflow to sync the dbt mission recordsdata and the DAG to an Amazon Easy Storage Service (Amazon S3) bucket.
In the course of the DAG run, dbt converts the fashions, checks, and macros to Amazon Redshift SQL statements, which run straight on the Redshift cluster.
If a process within the DAG fails, the DAG invokes an AWS Lambda perform to ship out a notification utilizing Amazon SNS.

Conditions

You should have the next conditions:

Create a dbt mission

A dbt mission is structured to facilitate modular, scalable, and maintainable information transformations. The next code is a pattern dbt mission construction that this submit will comply with:

MY_SAMPLE_DBT_PROJECT
├── .github
│   └── workflows
│       └── publish_assets.yml
└── src
    ├── dags
    │   └── dbt_sample_dag.py
    └── my_sample_dbt_project
        ├── macros
        ├── fashions
        └── dbt_project.yml

dbt makes use of the next YAML recordsdata:

dbt_project.yml – Serves as the principle configuration to your mission. Objects on this mission will inherit settings outlined right here until overridden on the mannequin stage. For instance:

# Identify your mission! Challenge names ought to include solely lowercase characters
# and underscores. 
title: 'my_sample_dbt_project'
model: '1.0.0'

# These configurations specify the place dbt ought to search for several types of recordsdata.
# The `model-paths` config, for instance, states that fashions on this mission might be
# discovered within the "fashions/" listing. 
model-paths: ["models"]
macro-paths: ["macros"]

# Configuring fashions
# Full documentation: https://docs.getdbt.com/docs/configuring-models
# On this instance config, we inform dbt to construct fashions within the instance/
# listing as views. These settings might be overridden within the particular person mannequin
# recordsdata utilizing the `{{ config(...) }}` macro.
fashions:
  my_sample_dbt_project:
    # Config indicated by + and applies to recordsdata underneath fashions/instance/
    instance:
      +materialized: view
      
on-run-end:
# add run outcomes to audit desk 
  - "{{ log_audit_table(outcomes) }}"

sources.yml – Defines the exterior information sources that your dbt fashions will reference. For instance:

sources:
  - title: sample_source
    database: sample_database
    schema: sample_schema
    tables:
      - title: sample_table

schema.yml – Outlines the schema of your fashions and information high quality checks. Within the following instance, we’ve outlined two columns, full_name for the mannequin model1 and sales_id for model2. Now we have declared them as the first key and outlined information high quality checks to examine if the 2 columns are distinctive and never null.

model: 2

fashions:
  - title: model1
    config: 
      contract: {enforced: true}

    columns:
      - title: full_name
        data_type: varchar(100)
        constraints:
          - kind: primary_key
        checks:
          - distinctive
          - not_null

  - title: model2
    config: 
      contract: {enforced: true}

    columns:
      - title: sales_id
        data_type: varchar(100)
        constraints:
          - kind: primary_key
        checks:
          - distinctive
          - not_null

Allow auditing inside dbt mission

Enabling auditing inside your dbt mission is essential for facilitating transparency, traceability, and operational oversight throughout your information pipeline. You may seize run metrics on the mannequin stage for every execution in an audit desk. By capturing detailed run metrics comparable to load identifier, runtime, and variety of rows affected, groups can systematically monitor the well being and efficiency of every load, rapidly establish points, and hint adjustments again to particular runs.

The audit desk consists of the next attributes:

load_id – An identifier for every mannequin run executed as a part of the load
database_name – The title of the database inside which information is being loaded
schema_name – The title of the schema inside which information is being loaded
title – The title of the article inside which information is being loaded
resource_type – The kind of object to which information is being loaded
execution_time – The time length taken for every dbt mannequin to finish execution as a part of every load
rows_affected – The variety of rows affected within the dbt mannequin as a part of the load

Full the next steps to allow auditing inside your dbt mission:

Navigate to the fashions listing (src/my_sample_dbt_project/fashions) and create the audit_table.sql mannequin file:

{%- set run_date = "CURRENT_DATE" -%}
{{
    config(
        materialized='incremental',
        incremental_strategy='append',
        tags=["audit"]
    )
}}

with empty_table as (
    choose
        'test_load_id'::varchar(200) as load_id,
        'test_invocation_id'::varchar(200) as invocation_id,
        'test_database_name'::varchar(200) as database_name,
        'test_schema_name'::varchar(200) as schema_name,
        'test_model_name'::varchar(200) as title,
        'test_resource_type'::varchar(200) as resource_type,
        'test_status'::varchar(200) as standing,
        solid('12122012' as float) as execution_time,
        solid('100' as int) as rows_affected,
        {{run_date}} as model_execution_date
)

choose * from empty_table
-- It is a filter so we'll by no means truly insert these values
the place 1 = 0

Navigate to the macros listing (src/my_sample_dbt_project/macros) and create the parse_dbt_results.sql macro file:

{% macro parse_dbt_results(outcomes) %}
    -- Create a listing of parsed outcomes
    {%- set parsed_results = [] %}
    -- Flatten outcomes and add to listing
    {% for run_result in outcomes %}
        -- Convert the run consequence object to a easy dictionary
        {% set run_result_dict = run_result.to_dict() %}
        -- Get the underlying dbt graph node that was executed
        {% set node = run_result_dict.get('node') %}
        {% set rows_affected = run_result_dict.get(
        'adapter_response', {}).get('rows_affected', 0) %}
        {%- if not rows_affected -%}
            {% set rows_affected = 0 %}
        {%- endif -%}
        {% set parsed_result_dict = {
                'load_id': invocation_id ~ '.' ~ node.get('unique_id'),
                'invocation_id': invocation_id,
                'database_name': node.get('database'),
                'schema_name': node.get('schema'),
                'title': node.get('title'),
                'resource_type': node.get('resource_type'),
                'standing': run_result_dict.get('standing'),
                'execution_time': run_result_dict.get('execution_time'),
                'rows_affected': rows_affected
                }%}
        {% do parsed_results.append(parsed_result_dict) %}
    {% endfor %}
    {{ return(parsed_results) }}
{% endmacro %}

Navigate to the macros listing (src/my_sample_dbt_project/macros) and create the log_audit_table.sql macro file:

{% macro log_audit_table(outcomes) %}
    -- depends_on: {{ ref('audit_table') }}
    {%- if execute -%}
        {{ print("Working log_audit_table Macro") }}
        {%- set run_date = "CURRENT_DATE" -%}
        {%- set parsed_results = parse_dbt_results(outcomes) -%}
         size  > 0 -%
            {% set allowed_columns = ['load_id', 'invocation_id', 'database_name', 
            'schema_name', 'name', 'resource_type', 'status', 'execution_time', 
            'rows_affected', 'model_execution_date'] -%}
            {% set insert_dbt_results_query -%}
                insert into {{ ref('audit_table') }}
                    (
                        load_id,
                        invocation_id,
                        database_name,
                        schema_name,
                        title,
                        resource_type,
                        standing,
                        execution_time,
                        rows_affected,
                        model_execution_date
                ) values
                    {%- for parsed_result_dict in parsed_results -%}
                        (
                            {%- for column, worth in parsed_result_dict.objects() %}
                                {% if column not in allowed_columns %}
                                    {{ exceptions.raise_compiler_error("Invalid
                                     column") }}
                                {% endif %}
                                 change("'", "''") %
                                '{{ sanitized_value }}'
                                {%- if not loop.final %}, {% endif %}
                            {%- endfor -%}
                        )
                        {%- if not loop.final %}, {% endif %}
                    {%- endfor -%}
            {%- endset -%}
            {%- do run_query(insert_dbt_results_query) -%}
        {%- endif -%}
    {%- endif -%}
    {{ return ('') }}
{% endmacro %}

Append the next traces to the dbt_project.yml file:

on-run-end:
  - "{{ log_audit_table(outcomes) }}"

Create a GitHub Actions workflow

This step is optionally available. In case you choose, you may skip it and as a substitute add your recordsdata on to your S3 bucket.

The next GitHub Actions workflow automates the deployment of dbt mission recordsdata and DAG file to Amazon S3. Change the placeholders {s3_bucket_name}, {account_id}, {role_name}, and {area} together with your S3 bucket title, account ID, IAM position title, and AWS Area within the workflow file.

To reinforce safety, it’s beneficial to make use of OpenID Join (OIDC) for authentication with IAM roles in GitHub Actions as a substitute of counting on long-lived entry keys.

title: Sync dbt Challenge with S3

on:
  workflow_dispatch:
  push:
    branches: [ main ]
    paths:
      - "src/**"

permissions:
  id-token: write   # That is required for requesting the JWT
  contents: learn    # That is required for actions/checkout
  pull-requests: write

jobs:
  sync-dev:
    runs-on: ubuntu-latest
    surroundings: dev
    defaults:
      run:
        shell: bash
    steps:
      - makes use of: actions/checkout@v4
      - title: Assume AWS IAM Position
        makes use of: aws-actions/[email protected]
        with:
          aws-region: {area}
          role-to-assume: arn:aws:iam::{account_id}:position/{role_name}
          role-session-name: my_sample_dbt_project_${{ github.run_id }}
          role-duration-seconds: 3600 # 1 hour

      - run: aws sts get-caller-identity

      - title: Sync dbt Mannequin recordsdata
        id: dbt_project_files
        working-directory: src/my_sample_dbt_project
        run: aws s3 sync . s3://{s3_bucket_name}/dags/dbt/my_sample_dbt_project 
        --delete
        continue-on-error: false

      - title: Sync DAG recordsdata
        id: dag_file
        working-directory: src/dags
        run: aws s3 sync . s3://{s3_bucket_name}/dags

GitHub has the next safety necessities:

Department safety guidelines – Earlier than continuing with the GitHub Actions workflow, make sure that department safety guidelines are in place. These guidelines implement required standing checks earlier than merging code into protected branches (comparable to primary).
Code assessment tips – Implement code assessment processes to ensure adjustments bear assessment. This will embrace requiring at the least one approving assessment earlier than code is merged into the protected department.
Incorporate safety scanning instruments – This may also help detect vulnerabilities in your repository.

Ensure you are additionally adhering to dbt-specific safety greatest practices:

Take note of dbt macros with variables and validate their inputs.
When including new packages to your dbt mission, consider their safety, compatibility, and upkeep standing to ensure they don’t introduce vulnerabilities or conflicts into your mission.
Assessment dynamically generated SQL to safeguard towards points like SQL injection.

Replace the Amazon MWAA occasion

Full the next steps to replace the Amazon MWAA occasion:

Set up the Cosmos library on Amazon MWAA by including astronomer-cosmos within the necessities.txt file. Ensure to examine for model compatibility for Amazon MWAA and the Cosmos library.
Add the next entries in your startup.sh script:
1. Within the following code, DBT_VENV_PATH specifies the placement the place the Python digital surroundings for dbt can be created. DBT_PROJECT_PATH factors to the placement of your dbt mission inside Amazon MWAA.
```
#!/bin/sh
export DBT_VENV_PATH="${AIRFLOW_HOME}/dbt_venv"
export DBT_PROJECT_PATH="${AIRFLOW_HOME}/dags/dbt"
```
2. The next code creates a Python digital surroundings on the path ${DBT_VENV_PATH} and installs the dbt-redshift adapter to run dbt transformations on Amazon Redshift:
```
python3 -m venv "${DBT_VENV_PATH}"
${DBT_VENV_PATH}/bin/pip set up dbt-redshift
```

Create a dbt person in Amazon Redshift and retailer credentials

To create dbt fashions in Amazon Redshift, you need to arrange a local Redshift person with the required permissions to entry supply tables and create new tables. It’s important to create separate database customers with minimal permissions to comply with the precept of least privilege. The dbt person shouldn’t be granted admin privileges, as a substitute, it ought to solely have entry to the precise schemas required for its duties.

Full the next steps:

Open the Amazon Redshift console and join as an admin (for extra particulars, check with Connecting to an Amazon Redshift database).
Run the next command within the question editor v2 to create a local person, and word down the values for dbt_user_name and password_value:
```
create person {dbt_user_name} password 'sha256|{password_value}';
```
Run the next instructions within the question editor v2 to grant permissions to the native person:
1. Hook up with the database the place you need to supply tables from and run the next instructions:
```
grant utilization on schema {schema_name} to {dbt_user_name};
grant choose on all tables in schema {schema_name} to {dbt_user_name};
```
2. To permit the person to create tables inside a schema, run the next command:
```
grant create on schema {schema_name} to {dbt_user_name};
```
Optionally, create a secret in AWS Secrets and techniques Supervisor and retailer the values for dbt_user_name and password_value from the earlier step as plaintext:

{
    "username":"dbt_user_name",
    "password":"password_value"
}

Making a Secrets and techniques Supervisor entry is optionally available, however beneficial for securely storing your credentials as a substitute of hardcoding them. To be taught extra, check with AWS Secrets and techniques Supervisor greatest practices.

Create a Redshift connection in Amazon MWAA

We create one Redshift connection in Amazon MWAA for every Redshift database, ensuring that every information pipeline (DAG) can solely entry one database. This strategy gives distinct entry controls for every pipeline, serving to forestall unauthorized entry to information. Full the next steps:

Log in to the Amazon MWAA UI.
On the Admin menu, select Connections.
Select Add a brand new document.
For Connection Id, enter a reputation for this connection.
For Connection Sort, select Amazon Redshift.
For Host, enter the endpoint of the Redshift cluster with out the port and database title (for instance, redshift-cluster-1.xxxxxx.us-east-1.redshift.amazonaws.com).
For Database, enter the database of the Redshift cluster.
For Port, enter the port of the Redshift cluster.

Arrange an SNS notification

Organising SNS notifications is optionally available, however they could be a helpful enhancement to obtain alerts on failures. Full the next steps:

Create an SNS matter.
Create a subscription to the SNS matter.
Create a Lambda perform with the Python runtime.
Modify the perform code in your Lambda perform, and change {topic_arn} together with your SNS matter Amazon Useful resource Identify (ARN):

import json

sns_client = boto3.consumer('sns')

def lambda_handler(occasion, context):
     strive:
        # Extract DAG title from occasion
        failed_dag = occasion['dag_name']
        
        # Ship notification 
        sns_client.publish(
            TopicArn={topic_arn}, 
            Topic="Knowledge modelling dags - WARNING", 
            Message=json.dumps({'default': json.dumps(f"Knowledge modelling DAG - 
            {failed_dag} has failed, please inform the information modelling workforce")}),
            MessageStructure="json"
        )
        
    besides KeyError as e:
        # Deal with lacking 'dag_name' within the occasion
        logger.error(f"KeyError: invalid payload - dag_name not current")

Configure a DAG

The next pattern DAG orchestrates a dbt workflow for processing and auditing information fashions in Amazon Redshift. It retrieves credentials from Secrets and techniques Supervisor, runs dbt duties in a digital surroundings, and sends an SNS notification if a failure happens. The workflow consists of the next steps:

It begins with the audit_dbt_task process group, which creates the audit mannequin.
The transform_data process group executes the opposite dbt fashions, excluding the audit-tagged one. Contained in the transform_data group, there are two dbt fashions, model1 and model2, and every is adopted by a corresponding take a look at process that runs information high quality checks outlined within the schema.yml file.
To correctly detect and deal with failures, the DAG features a dbt_check Python process that runs a customized perform, check_dbt_failures. That is essential as a result of when utilizing DbtTaskGroup, particular person model-level failures contained in the group don’t routinely propagate to the duty group stage. In consequence, downstream duties (such because the Lambda operator sns_notification_for_failure) configured with trigger_rule="one_failed" is not going to be triggered until a failure is explicitly raised.

The check_dbt_failures perform addresses this by inspecting the outcomes of every dbt mannequin and take a look at, and elevating an AirflowException if a failure is discovered. When an AirflowException is raised, the sns_notification_for_failure process is triggered.

If a failure happens, the sns_notification_for_failure process invokes a Lambda perform to ship an SNS notification. If no failures are detected, this process is skipped.

The next diagram illustrates this workflow.

Configure DAG variables

To customise this DAG to your surroundings, configure the next variables:

project_name – Ensure the project_name matches the S3 prefix of your dbt mission
secret_name – Present the title of the key that shops dbt person credentials
target_database and target_schema – Replace these variables to mirror the place you need to land your dbt fashions in Amazon Redshift
redshift_connection_id – Set this to match the connection configured in Amazon MWAA for this Redshift database
sns_lambda_function_name – Present the Lambda perform title to ship SNS notifications
dag_name – Present the DAG title that can be handed to the SNS notification Lambda perform

import os
import json
import boto3
from airflow import DAG
from cosmos import (
    DbtTaskGroup, ProfileConfig, ProjectConfig,
    ExecutionConfig, RenderConfig
)
from cosmos.constants import ExecutionMode, LoadMode
from cosmos.profiles import RedshiftUserPasswordProfileMapping
from pendulum import datetime
from airflow.operators.python_operator import PythonOperator
from airflow.suppliers.amazon.aws.operators.lambda_function import (
    LambdaInvokeFunctionOperator
)
from airflow.exceptions import AirflowException

# mission title - ought to match the s3 prefix of your dbt mission
project_name = "my_sample_dbt_project"
# title of the key that shops dbt person credentials 
secret_name = "dbt_user_credentials_secret"
# goal database to land dbt fashions
target_database = "sample_database"
# goal schema to land dbt fashions
target_schema = "sample_schema"
# Redshift connection title from MWAA
redshift_connection_id = "my_sample_dbt_project_connection"
# sns lambda perform title
sns_lambda_function_name = "sns_notification"
# dag title - this can be handed to SNS for notification
payload = json.dumps({
            "dag_name": "my_sample_dbt_project_dag"
        })

Incorporate DAG elements

After setting the variables, now you can incorporate the next elements to finish the DAG.

Secrets and techniques Supervisor

The DAG retrieves dbt person credentials from Secrets and techniques Supervisor:

sm_client = boto3.consumer('secretsmanager')

def get_secret(secret_name):
    strive:
        get_secret_value_response = sm_client.get_secret_value(SecretId=secret_name)
        return json.masses(get_secret_value_response["SecretString"])
    besides Exception as e:
        elevate

secret_value = get_secret(secret_name)
username = secret_value["username"]
password = secret_value["password"]

Redshift connection configuration

It makes use of RedshiftUserPasswordProfileMapping to authenticate:

profile_config = ProfileConfig(
    profile_name="redshift",
    target_name=target_database,
    profile_mapping=RedshiftUserPasswordProfileMapping(
        conn_id=redshift_connection_id,
        profile_args={"schema": target_schema,
                      "person": username, "password": password}
    ),
)

dbt execution setup

This code accommodates the next variables:

dbt executable path – Makes use of a digital surroundings
dbt mission path – Is positioned within the surroundings variable DBT_PROJECT_PATH underneath your mission

execution_config = ExecutionConfig(
    dbt_executable_path=f"{os.environ['DBT_VENV_PATH']}/bin/dbt",
    execution_mode=ExecutionMode.VIRTUALENV,
)

project_config = ProjectConfig(
    dbt_project_path=f"{os.environ['DBT_PROJECT_PATH']}/{project_name}",
)

Duties and execution circulation

This step contains the next elements:

Audit dbt process group (audit_dbt_task) – Runs the dbt mannequin tagged with audit
dbt process group (transform_data) – Runs the dbt fashions tagged with operations, excluding the audit mannequin

In dbt, tags are labels that you may assign to fashions, checks, seeds, and different dbt assets to prepare and selectively run subsets of your dbt mission. In your render_config, you will have exclude=["tag:audit"]. This implies dbt will exclude fashions which have the tag audit, as a result of the audit mannequin runs individually.

Failure examine (dbt_check) – Checks for dbt mannequin failures, raises an AirflowException if upstream dbt duties fail
SNS notification on failure (sns_notification_for_failure) – Invokes a Lambda perform to ship an SNS notification upon a dbt process failure (for instance, a dbt mannequin within the process group)

def check_dbt_failures(**kwargs):
    if kwargs['ti'].state == 'failed':
        elevate AirflowException('Failure in dbt process group')

with DAG(
    dag_id="my_sample_dbt_project_dag",
    start_date=datetime(2025, 4, 2),
    schedule_interval="@every day",
    catchup=False,
    tags=["dbt"]
):

    audit_dbt_task = DbtTaskGroup(
        group_id="audit_dbt_task",
        execution_config=execution_config,
        profile_config=profile_config,
        project_config=project_config,
        operator_args={
            "install_deps": True,
        },
        render_config= RenderConfig(
            choose=["tag:audit"],
            load_method=LoadMode.DBT_LS
        )
    )

    transform_data = DbtTaskGroup(
        group_id="transform_data",
        execution_config=execution_config,
        profile_config=profile_config,
        project_config=project_config,
        operator_args={
            "install_deps": True,
            # set up crucial dependencies earlier than working dbt command
        },
        render_config= RenderConfig(
            exclude=["tag:audit"],
            load_method=LoadMode.DBT_LS
        )
    )

    dbt_check = PythonOperator(
        task_id='dbt_check', 
        python_callable=check_dbt_failures,
        provide_context=True,
    )

    sns_notification_for_failure = LambdaInvokeFunctionOperator(
        task_id="sns_notification_for_failure",
        function_name=sns_lambda_function_name,
        payload=payload,
        trigger_rule="one_failed"
    )

    audit_dbt_task >> transform_data >> dbt_check >> sns_notification_for_failure

The pattern dbt orchestrates a dbt workflow in Amazon Redshift, beginning with an audit process and adopted by a process group that processes information fashions. It features a failure dealing with mechanism that checks for failures and raises an exception to set off an SNS notification utilizing Lambda if a failure happens. If no failures are detected, the SNS notification process is skipped.

Clear up

In case you now not want the assets you created, delete them to keep away from further prices. This contains the next:

Amazon MWAA surroundings
S3 bucket
IAM position
Redshift cluster or serverless workgroup
Secrets and techniques Supervisor secret
SNS matter
Lambda perform

Conclusion

By integrating dbt with Amazon Redshift and orchestrating workflows utilizing Amazon MWAA and the Cosmos library, you may simplify information transformation workflows whereas sustaining sturdy engineering practices. The pattern dbt mission construction, mixed with automated deployments by way of GitHub Actions and proactive monitoring utilizing Amazon SNS, gives a basis for constructing dependable information pipelines. The addition of audit logging facilitates transparency throughout your transformations, so groups can preserve excessive information high quality requirements.

You should use this resolution as a place to begin to your personal dbt implementation on Amazon MWAA. The strategy we outlined emphasizes SQL-based transformations whereas incorporating important operational capabilities like deployment automation and failure alerting. Get began by adapting the configuration to your surroundings, and construct upon these practices as your information wants evolve.

For extra assets, check with Handle information transformations with dbt in Amazon Redshift and Redshift setup.

Concerning the authors

Cindy Li is an Affiliate Cloud Architect at AWS Skilled Providers, specialising in Knowledge Analytics. Cindy works with prospects to design and implement scalable information analytics options on AWS. When Cindy will not be diving into tech, yow will discover her out on walks along with her playful toy poodle Mocha.

Akhil B is a Knowledge Analytics Guide at AWS Skilled Providers, specializing in cloud-based information options. He companions with prospects to design and implement scalable information analytics platforms, serving to organizations remodel their conventional information infrastructure into fashionable, cloud-based options on AWS. His experience helps organizations optimize their information ecosystems and maximize enterprise worth by way of fashionable analytics capabilities.

Joao Palma is a Senior Knowledge Architect at Amazon Internet Providers, the place he companions with enterprise prospects to design and implement complete information platform options. He focuses on serving to organizations remodel their information into strategic enterprise belongings and enabling data-driven choice making.

Harshana Nanayakkara is a Supply Guide at AWS Skilled Providers, the place he helps prospects sort out complicated enterprise challenges utilizing AWS Cloud expertise. He focuses on information and analytics, information governance, and AI/ML implementations.