
10 Years of experience As an Senior Systems Administrator and Azure Engineer, the primary responsibilities involve overseeing the design, deployment, and management of Azure cloud infrastructure and services to meet organizational needs effectively. These roles requires a deep understanding of Azure's ecosystem and expertise in system administration, networking, security, and automation. As a Azure Senior Systems Administrator and Engineer i actively i am involved in designing, deploying, and maintaining Azure infrastructure solutions that enable organizations to leverage the full potential of cloud computing while ensuring security, reliability, and performance.
Databricks Admin
Platform Administrator:
Infrastructure Management: Oversee the provisioning, scaling, and management of databricks infrastructure including clusters, storage, and networking resources. Implemented security policies, access controls, and data protection measures to safeguard sensitive data and comply with regulatory requirements. Coordinated integrations with other systems, data sources, and services to enable seamless data workflows. Developed and maintained backup strategies and disaster recovery plans to ensure data resilience and business continuity.
DevOps/IT Operations:
Implemented CI/CD pipelines for deploying databricks notebooks, jobs, and applications across development, testing, and production environments.
Monitoring and Alerting: Set up monitoring tools and dashboards to track platform health, performance metrics, and job status, with automated alerts for critical events or anomalies.
Capacity Planning: Forecast resource requirements and plan capacity expansions or upgrades to meet growing demand and maintain service levels.
Provisioning and Optimization:
Provisioned Databricks Infrastructure using Terraform to manage the underlying infrastructure required for Databricks, such as virtual machines, networking components, and storage resources. This includes setting up virtual networks, subnets, security groups, and other resources needed to deploy Databricks clusters securely.
Orchestrated databricks clusters, where I meticulously tailor configurations to meet the diverse needs of our data practitioners. From choosing the optimal instance types to fine-tuning auto-scaling policies, every decision is aimed at striking the delicate balance between performance and cost efficiency. Regular optimization exercises ensure that resources are utilized judiciously, maximizing the value extracted from our investment in databricks.
Integration and Automation:
To navigate data, seamless integration with external systems. I engineer data pipelines that traverse disparate data sources, orchestrating a symphony of data movement orchestrated by databricks jobs and apache airflow. Automation scripts DevOps principles, streamline infrastructure provisioning and configuration, liberating valuable time for innovation and exploration.