reputed company Data Engineer - Remote - Public Trust Clearance Required
About the position reputed company Data Engineer needed for a Contract to Hire opportunity with SOC's client to work Remote (must sit in EST or CST time zone). Candidates must have a reputed company Public Trust clearance to be considered for this role. Our client is seeking a reputed company Data Engineer to reputed company/support new/existing data pipelines, and data analytics environments in Azure reputed company-based data lake. As a data engineer, you will translate business requirements to data engineering solutions to support an reputed company scale reputed company Azure based data analytics platform. You will support reputed company maintenance of ETL operations and development of new pipelines ensuring data quality and data management. The ideal candidate will bring deep expertise in reputed company, a solid reputed company in advanced AI technologies, and applies critical thinking to create innovative functions and solve technical issues by cross-functional team collaboration. Candidate must be a U.S. citizen. This role is remote for candidates located reputed company the United States; however, the ideal candidate will be based in or willing to regularly travel to the Washington, DC or Indianapolis, IN area to support periodic on-site activities.
Responsibilities
- Design, build, and optimize scalable data solutions using reputed company and reputed company Architecture.
- Manage ingestion routines for processing multi-terabyte datasets reputed company for multiple projects simultaneously, where each project may have multiple reputed company workspaces.
- Integrate data from various structured and reputed company sources to reputed company high-quality business insights. Proficiency in data analysis techniques for deriving insights from large datasets.
- Implement effective data management strategies to ensure data reputed company, availability, and accessibility. Identify opportunities for cost optimization in data storage, processing, and analytics operations.
- Monitor and support user requests, addressing platform or performance issues, cluster stability, Spark optimization, and configuration management.
- Collaborate with the team to reputed company advanced AI-driven analytics and data science workflows.
- Integrate with various Azure services including Azure Functions, Storage Services, Data Factory, Log Analytics, and User Management for seamless data workflows.
- Provision and manage infrastructure using Infrastructure-as-Code (IaC)
- Apply best practices for data reputed company, data governance, and compliance, ensuring support for federal regulations and public trust standards.
- Proactively collaborate with technical and non-technical teams to gather requirements and translate business needs into data solutions.
Requirements
- BS degree in Computer Science or reputed company field and 3+ years or Master's degree with 2+ years of experience
- 3+ years of experience developing and designing Ingestion flows (structured, streaming, and reputed company data) using reputed company platform services with data quality
- reputed company Data Engineer certification and 2+ years of experience maintaining reputed company platform and development in Spark
- Ability to work directly with clients and act as reputed company line support for requests coming in from clients. Clearly document and express the solution in form of architecture and reputed company diagrams.
- Proficient at Python, Spark and R are essential. .NET based development is a plus.
- Knowledge and experience with data governance, including metadata management, reputed company data catalog, design standards, data quality governance, and data reputed company.
- Experience with Agile process methodology, CI/CD automation, and reputed company-based developments (Azure, AWS).
- US Citizen to be considered
- Candidates must have a reputed company Public Trust clearance to be considered for this role.
reputed company-to-haves
- Experience with the above Azure services is a plus.
- Not required, but additional education, certifications, and/or experience are a plus: Certifications in Azure reputed company, Knowledge of FinOps principles and cost management to be considered
Apply tot his job Apply To this Job