Google provides $300 free Trial period which lasts for 90 days. I activated my free trial and started looking/learning around in Google Cloud Platform (GCP) console.
FYI, this is the resource hierarchy in 3 cloud service providers.
Data is needed for testing any use cases. We can either upload the data files directly to the Google storage account or build a process to transfer the data from other cloud…
In this blog post, you shall see the process involved in migrating a database from Azure SQL to AWS RDS for SQL Server using AWS DMS.
The process involves these steps.
AWS provides a tool called Schema…
In my earlier blog post, I explained the process for copying a file from AWS S3 bucket to ADLS Gen2 storage account using ADF.
We may want the copy pipeline to run only when there is a file in AWS S3 bucket. This will save the Data Integration Unit (DIU) in ADF.
We can achieve this in various ways. I achieved this using
Azure Logic Apps
Azure Functions is a serverless compute…
An app team in our company reached out to us asking for help in designing a reporting solution hosted in Azure. Their business requirements are
We can go with different design approaches for this requirement. But, the challenge lies in choosing the optimal approach which addresses all the business requirements and probably costs less.
In this blog post, you can know about the different design approaches we developed for the app team.
Design approach 1:
Azure provides different database migration services to handle different Rs of cloud adoption rationalization.
Note: AWS also provides a service with the same name Database Migration Service (DMS), for migrating databases to AWS data platform.
I used Azure DMS to migrate an…
The process to copy a file from AWS S3 bucket to Azure Data Lake Storage (ADLS) Gen2 storage account using Azure Data Factory (ADF) is easy to implement.
It involves these steps.
I used NYC taxi FHV public dataset file for the purpose…
Azure Analysis Services is a fully managed platform as a service (PaaS) that provides enterprise-grade data models in the cloud. It integrates with many Azure services enabling you to build sophisticated analytics solutions.
AAS is the cloud equivalent of SQL Server Analysis Services (SSAS).
I did a PoC on AAS, deploying data models from 3 data sources to AAS instance.
Data sources are
In this blog post, I tried to explain the process of deploying the data models to AAS.
Azure Bot Service provides the core components for creating bots, including the Bot Framework SDK for developing bots and the bot service for connecting bots to channels.
We are can create the bot in 2 ways.
1. Create Web App Bot in Azure portal, download the source code and develop the bot locally.
This approach would need coding skills.
2. Use QnA Maker or Luis for creating the bot.
This approach doesn’t need coding skills.
I chose Azure data store selection as my use case for the bot. I see that a bot can address many use cases within the…
SQL Data Sync is a service built on Azure SQL Database that lets you synchronize the data bi-directionally across multiple databases, both on-premises and in the cloud.
Data Sync uses a hub and spoke topology to synchronize data. You define one of the databases in the sync group as the hub database. The rest of the databases are member databases. Sync occurs only between the hub and individual members.
ETL pipelines require many layers of data movement resulting in much operational complexity, and performance impact on the transactional workloads in Cosmos DB.
For generating reports in Power BI, we have to first perform Flattening and transforming JSON documents.
We need to maintain and manage separate setup for ETL, reporting and machine learning activities on the operational data in Cosmos DB. Also, all these activities will have direct impact on RU usage.
This is where Synapse Link and Synapse Analytics fit in and make our work less cumbersome, without impacting the RU usage.