Sign in

Databases enthusiast

Google provides $300 free Trial period which lasts for 90 days. I activated my free trial and started looking/learning around in Google Cloud Platform (GCP) console.

FYI, this is the resource hierarchy in 3 cloud service providers.

  1. Azure: Root Management group → Management groups → Subscriptions → Resource groups → Resources
  2. AWS: Root → Organizations → Member Accounts → Resources
  3. GCP: Organization → Folders → Projects → Resources

In my earlier blog post, I explained the process to migrate a database from AWS RDS to Azure SQL using Azure DMS.

In this blog post, you shall see the process involved in migrating a database from Azure SQL to AWS RDS for SQL Server using AWS DMS.

The process involves these steps.

  1. Create Azure SQL single database instance.
  2. Create Amazon RDS for SQL Server database instance.
  3. Create AWS Data Migration Service (DMS) replication instance.
  4. Create database migration tasks.
  5. Perform database migration from Azure SQL to Amazon RDS for SQL Server using AWS DMS.

In my earlier blog post, I explained the process for copying a file from AWS S3 bucket to ADLS Gen2 storage account using ADF.

We may want the copy pipeline to run only when there is a file in AWS S3 bucket. This will save the Data Integration Unit (DIU) in ADF.

We can achieve this in various ways. I achieved this using

  1. Azure Logic Apps
  2. Get Metadata and If condition in ADF

Azure provides many integration and automation services. Of them, my favorite are Azure Logic Apps and Azure Functions.

Azure Functions is a serverless compute…

An app team in our company reached out to us asking for help in designing a reporting solution hosted in Azure. Their business requirements are

  1. Send data from on-prem PostgreSQL database to Azure on daily basis.
  2. Build reporting solution on top of the data sent from on-prem database.

In this blog post, you can know about the different design approaches we developed for the app team.

Design approach 1:

In this…

Azure provides different database migration services to handle different Rs of cloud adoption rationalization.

  1. Azure Migrate for Rehost (lift & shift) — both source and target servers will have the same database (E.g., on-prem SQL Server 2012 migrating to SQL Server 2012 on Azure VM)
  2. Database Migration Service (DMS) for Refactor and Rearchitect — source database will be upgraded on the target server (E.g., on-prem SQL Server 2012 migrating to Azure SQL)

I used Azure DMS to migrate an…

The process to copy a file from AWS S3 bucket to Azure Data Lake Storage (ADLS) Gen2 storage account using Azure Data Factory (ADF) is easy to implement.

It involves these steps.

  1. Create AWS S3 bucket
  2. Get Access key ID and Secret access key to access AWS S3 bucket
  3. Create ADLS Gen2 storage account
  4. Create linked services for AWS S3 and ADLS Gen2 in ADF
  5. Create datasets for AWS S3 and ADLS Gen2 in ADF
  6. Create pipeline in ADF for copying the file from AWS S3 to ADLS Gen2

Azure Analysis Services is a fully managed platform as a service (PaaS) that provides enterprise-grade data models in the cloud. It integrates with many Azure services enabling you to build sophisticated analytics solutions.

AAS is the cloud equivalent of SQL Server Analysis Services (SSAS).

I did a PoC on AAS, deploying data models from 3 data sources to AAS instance.

Data sources are

  1. Azure SQL (PaaS)
  2. SQL VM (IaaS)
  3. On-prem SQL Server

Azure Bot Service provides the core components for creating bots, including the Bot Framework SDK for developing bots and the bot service for connecting bots to channels.

We are can create the bot in 2 ways.

1. Create Web App Bot in Azure portal, download the source code and develop the bot locally.

This approach would need coding skills.

2. Use QnA Maker or Luis for creating the bot.

This approach doesn’t need coding skills.

I chose Azure data store selection as my use case for the bot. I see that a bot can address many use cases within the…

SQL Data Sync is a service built on Azure SQL Database that lets you synchronize the data bi-directionally across multiple databases, both on-premises and in the cloud.

Data Sync uses a hub and spoke topology to synchronize data. You define one of the databases in the sync group as the hub database. The rest of the databases are member databases. Sync occurs only between the hub and individual members.

  • Sync Metadata Database contains the metadata and log for Data Sync. …

ETL pipelines require many layers of data movement resulting in much operational complexity, and performance impact on the transactional workloads in Cosmos DB.

For generating reports in Power BI, we have to first perform Flattening and transforming JSON documents.

We need to maintain and manage separate setup for ETL, reporting and machine learning activities on the operational data in Cosmos DB. Also, all these activities will have direct impact on RU usage.

This is where Synapse Link and Synapse Analytics fit in and make our work less cumbersome, without impacting the RU usage.

Value addition with Azure Synapse Link

Bharath Nunepalli

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store