Designed and built an end-to-end latency pipeline dashboard, coupled with Argus metrics, for monitoring data flow latency into Splunk at each system component.
Reduced the mean time to notice the lag and allowed on-call personnel to identify rapidly the component responsible. This eliminated the need for the human effort of looking into each component to confirm its eligibility during the latency.
Splunk Contributions
Falcon
Design
Active Splunk contributor in Falcon Architecture and Design
Splunk multi-stack architecture was achieved by designing spinnaker bootstrap and deploy pipelines
Splunk multi-site deployment method designed to overcome restricted cap-add circumstances
Development
SME in Terraform and Spinnaker - mastered
Developed terraform modules in Falcon to manage Splunk infrastructure and setup scripts
Created master-child spinnaker pipelines in Falcon for deploying Splunk components
Developed spot cluster pipelines for individual feature testing
Enabled mutual-TLS on various Splunk communication routes
Splunk functioning with third-party certificates was understood, and custom scripts were built to accomplish the same
Multi-site deployment logic for robust indexer clustering over several indexer islands was developed and tested
Legacy Spinnaker terraform pipes were upgraded to BOM-driven pipelines. This release procedure has been devised to provide a smooth transfer of current spot and master pipelines.
Created workflows and scripts for uploading secrets/certificates/licenses into vault
Worked on setting up Splunk to operate as a systemd service
Investigated current Falcon Architecture and developed Terraform modules for NLB and Route53 resource deployments
Developed knowledge in a variety of AWS services and resources
CI/CD Automation
Actively engaged in design modifications necessary to improve the patching model, as well as reviewed code changes
Designed and implemented a code review and release procedure
FIT tests are being updated in order to provide component-specific validations
VMF work-flow has been updated to include both commercial and blackjack deployments through VMF
Changes that support continuous deployment
Service Ownership(Team & Customer Support)
Taking part in knowledge-sharing workshops. Several discussions regarding prospective design modifications and additional features were held
Have been a proponent of several design changes/considerations
Supported other team members comprehend Splunk and Madeira design decisions by providing details on how they are setup
Design choices, work-flows, new features, and testings were all documented. On runbooks, we worked together.
Participated in a number of Splunk problem investigations and collaborated with Splunk.com support staff
Assisted customers/users in understanding Splunk in Falcon work flow and resolving reported issues
BlackJack
Leading Splunk in Blackjack initiative
Release captain for deploying Splunk & Madeira services in SDev1 & SDev2
Design changes and Development
Blackjack design and architecture papers were studied, and Splunk and Madeira deployment and maintenance models were adjusted to meet blackjack needs
Blackjack SDev2 and entry criteria items were successfully finished, and exit criteria items are now being worked on
Terraform and Spinnaker code bases have been updated to provide both commercial and blackjack functioality
Completed complete deployment and operations runbooks for blackjack from the ground up
Worked in cross-team contexts to comprehend design modifications to other blackjack services
Worked on automating the management of FIT secrets and avoiding the present manual process of sending FIT credentials to the FIT team
Collaboration with the VMF team to understand VMF flow in blackjack and make appropriate adjustments for Splunk and Madeira VMF deployments in blackjack settings
I collaborated with the Ajna/Mirus team to simulate data production in blackjack and evaluate E2E data flow
DnR integration
Splunk indexer clustering was designed to work withDnR
DnR-specific Splunk settings have been added to allow DnR logs to be onboarded into Splunk
Deployed and maintained a spot cluster in dev1 to assist DnR integration efforts
Regular syncup calls with DnR team
Provided technical support on blackjack questions
SDev2 support
Supporting blackjack operators to onboard Splunk and Madeira in SDev2
