Job Description Role – Data Engineer Location –Remote Fulltime/FTE Salary – $95000/Annum --- $115000/Annum
Job Description Must Have Technical/Functional Skills
Solid understanding of search technology, real-time data pipeline construction, ETL processes, and various programming languages including Java and Python. Proficient with UNIX/Linux, scheduling, and orchestration tools.
Expert in developing robust Python scripts for automating ETL tasks, data ingestion, cleansing, validation, and workflow orchestration.
Experienced in implementing RESTful APIs in Python for data exposure and consumption, and skilled in using Python libraries (e.g., Pandas, NumPy, SQLAlchemy) for data manipulation and database connectivity.
Design and deploy microservices and data applications on Kubernetes clusters, managing deployments, services, and autoscaling.
Implement and manage CI/CD pipelines that automate the deployment of Docker images to Kubernetes environments.
Create optimized Docker files to package data applications (Python scripts, ETL services) and their dependencies, ensuring consistent environments.
Customize deployments using values.yaml in Helm for different environments (dev, QA, production).
Roles & Responsibilities
Configure and troubleshoot authentication and authorization using Kerberos to secure access to Big Data systems like Hadoop, Spark, or specific databases. Implement security principles and role-based access in data environments.
Perform basic database administration tasks such as performance monitoring, indexing, and query optimization on engines like SQL Server, PostgreSQL, Oracle, and Greenplum.
Efficiently extract, transform, and load data between these heterogeneous databases.
Requires a Bachelors in Computer Science, Computer Engineering or related field and some experience ADO/GIT, ETL, SQL, UNIX/Linux, Docker/Kubernetes, API Development, JSON, Kafka, Automated Testing, BDD, Big Data distributed systems, various programming languages like Java and Python, orchestration tools and processes or other directly related experience.
Develops, tests, and modifies software to enhance data platform and application efficiency. Provides technical support for issues.
Actively participates in agile ceremonies, including program increment planning, daily standups, team backlog grooming, iteration retrospectives, team demos, and inspect & adapt sessions.
Supports test and QA efforts on data projects and coordinates with data operations teams for production deployments.
Possesses strong analytical, organizational, and problem-solving skills. Demonstrates the ability and desire to quickly learn new technologies and adapt to changing technology and priorities.
Excellent verbal and written communication skills with both technical and non-technical staff. Capable of working independently, handling multiple concurrent projects, prioritizing effectively, and collaborating effectively within a team environment.
Skilled at eliciting, gathering, analyzing user requirements, and interpreting, validating, and mapping business requirements to appropriate solutions.
Able to meet deadlines.
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
