Azure Data Factory is a cloud-based data orchestration service that enables data movement and transformation. You can set up code repository for Azure Data Factory (ADF) and have an end to end integrated development and release experience. Attach to a code repository for data factory and have your configuration JSON for the dataset, linked services, and pipelines. In this post, we will quickly look how to set up the code repository for Azure Data Factory.
Code Repository for Azure Data Factory
From the Azure Portal, open the respective Azure Data Factory resource. Select the “Author & Monitor” tile under quick links for open the ADF Pipeline. Consider you have an existing ADF Pipeline. Select the “Set up Code Repository” option to open the repository settings wizard.
Repository settings wizard will have the option to provide the information related with data factory and code repository. At this point of type, we have the only support of Visual Studio Team Services Git as a repository.
Select the respective projects and details of the branch, save the information. Now you can create a new local branch, however, for now, let’s go ahead by selecting “master” branch.
That’s it. Your repository configuration is done. You can go back to your Visual Studio Team Services, to check the pushed code.
Now, VSTS code repository now has all the JSON files for the dataset, linked services, and pipeline. You can now start doing your changes, commit it to push it back to your Azure Data Factory.
From the Visual Studio Team services, you can check out all the changes history including the changes from the pipeline.
From here onward, you can have your code with your Favorite IDE like Visual Studio or Visual Studio code to do necessary changes for your Azure Data Factory.
Local Branch for ADF and Create Pull Request
You can create pull requests and have your local branch for Azure Data Factory. Maintain your changes in the local branches and push the code to master using a pull request.
This will enable end to end code flow management for proper release management.
