ā¢ Designed, deployed, monitored and maintained highly scalable distributed data processing platform on Aws Eks. ā¢ Designed and deployed 10 plus different services on Kubernetes , building images / modifying required kubernetes artifacts , provisioning eks clusters node groups. ā¢ Hands on development work refactoring existing open source code base for enhancements and features using python , scala , java. ā¢ Design of new rbac controls based AWS Imdsv2 security requirements on 12 different clusters with 2500 instances. ā¢ Automated the deployment and management of auto scaled Aws Eks cloud Infrastructure. ā¢ Created CICD Pipeline using Spinnaker Helm charts Pulumi Iac for deployment of batch and interactive data processing Kubernetes resources. ā¢ Optimizing Troubleshooting data platform which runs 100k spark jobs with size of 3000 plus executors with Spark on K8s platform processing petabyte scale data sets. ā¢ Build Kube manifests, Helm Charts, Docker images and related automation using Ias to deploy the resources. ā¢ Triage issues as they arise, create strategies for long term permanent fixes to critical production incidents. ā¢ Developed Airflow DAG scripts data pipeline for launching batch jobs on EKS. ā¢ Continuously improve the observability of the tech stack by integrating logging, testing, metrics, tracing, dashboards, and alerts with CD pipelines using Data Dog, Splunk , Grafana. ā¢ Participate in 24x7 on-call rotations for Apple Siri AIML data platform stack , which includes Kubernetes , On-prem , Trino , Airflow , Jupyterhub.