This site uses cookies. To find out more, see our Cookies Policy

Service Operations Engineer - Master in San Francisco, CA at Integrated Resources, Inc

Date Posted: 1/11/2019

Job Snapshot

Job Description

Position Overview:
Today we have Petabytes of Weather, Satellite Imagery and Planting and Harvest machine generated data. Our data is stored in S3, DynamoDB, RDS, Spark, ElasticSearch. We have a number of ingest pipelines for Weather, Satellite and machine data.

Client is confronted with the challenges of how to manage, maintain and secure massive data sets. For example how do you go about archiving 20 billion S3 objects. We are challenged with picking the most appropriate stack for the task when several options are available. We are challenged by scalable performance and tradeoffs with cost. In the future our datasets will expand by going global supporting longer time Clients and more detailed resolution.

The successful candidate will be responsible for operational tasks such as data inventory management; data lifecycle management (backups, archiving); data security, access and auditing; data quality, database capacity planning, tuning and performance. The candidate will be required to know how to leverage existing services on AWS. The candidate can articulate to engineering teams when one technology is superior to another for a specific task. For example DynamoDB vs RDS.
Our team's challenge is to accelerate The Clients's engineering organization's innovation and research by delivering a highly scalable, reliable, available and secure cloud infrastructure. Your challenge - should you decide to accept it - is to collaborate with us to build this better and secure world for our scientists and engineers.
What you will do:
Design, build and deploy secure and compliant SQL and NoSQL databases from determining business requirements to planning and deploying staging, test and production systems in our public cloud environment.
Empower our engineering and science teams to provision, scale, maintain and secure databases for R&D environments through automated workflows.
Architect our database systems to be more reliable, secure, faster, and make best use of available infrastructure.
Collaborate with product managers and leadership to drive database system requirements and delivering the maximum value to all engineering and science groups
Proactively identify security flaws and vulnerabilities, and conduct security reviews for all of our database systems.
Lead provisioning of databases, upgrades, patching, environment cloning, backup/recovery and restore, monitoring, capacity planning and perform failovers.
Datastores centralized logging enhancements (audit trails)
Enhance datastores security using monitoring, service health dashboards, operational playbooks
Implement automated data governance (GDPR, PII) and management processes.
Evangelize best IAM practices and security of Climate.com's datastores.
Support Security Audits, PII, GDPR requests (Databases).

Basic Qualifications:
Bachelor's degree
2+ years of knowledge in at least one programming language (Python or GoLang)
Experience with Service Oriented Architectures (SOA)
Demonstrable basic knowledge of TCP/IP, HTTP, application security, and experience supporting multi-tier web application architectures.
Familiarity with the security frameworks and application security best practices.
Attention to detail and a safe pair of hands.
5+ years administering large scale database environments (10+ custers)
8+ years of expert database engineer of experience.
Strong written and verbal communication skills. Specific relevant experience should include writing and presenting application security assessment reports. Candidate should have experience making and defending sound technical arguments that incorporate relevant technical and business considerations, and building consensus among stakeholders

Preferred Qualifications:
Knowledge of RDS, Redshift and DynamoDB databases in Amazon.
Experience with any public cloud based provider such as Amazon Web Services, Microsoft Azure, or Google Cloud Compute.
Proficiency in a Unix/Linux environment
Familiarity with distributed data platforms (e.g. DynamoDB, Hadoop, EMR, Spark and PostGIS)
Maturity, judgement, negotiation/influence, analytical, and leadership skills.
MS or PhD in Computer Science.