Technical Skills
Databricks Automation: Used Python and Databricks to develop Automated Job creation. Improved communication efficiency and automated repetitive tasks, saving time for other work.
Reduced Cost: worked on optimizing the cluster and reduced cluster cost. Used vaccum optimization for datalake storage that reduced storage cost by significant amount.
Unity Catalog Migration: Helped on migrating the data from hive_metastore to Unity Catalog. Converted the data to delta format from parquet.
Soft Skills
Team Collaboration: Collaborated with Infosys to modernize the codebase. Worked with other departments to implement robust infrastructure. Coordinated with development and operations teams.
Problem Solving: Identified inefficiencies in the legacy data stored in parquet and worked on converting it to delta format. Implemented automations to handle repetitive tasks.
Fast Learning: Quickly learned the infrastructure setup. Learned technologies to implement workflows and various new technologies.
Adaptability: Adapting to new technologies, processes, and environments.
Planning: Planned migrations, designed code architectures, and wrote detailed documentation. Managed tasks using Service Now Portal.
Communication: Communicated with mutiple teams for codebase migration. Coordinated with various teams and departments. Clearly communicated technical ideas and collaborated effectively with team members.