Skip to content

Instantly share code, notes, and snippets.

@yashk
Created April 5, 2017 06:38
Show Gist options
  • Save yashk/40b4093c55fccf247fbb83a1090b9010 to your computer and use it in GitHub Desktop.
Save yashk/40b4093c55fccf247fbb83a1090b9010 to your computer and use it in GitHub Desktop.
Tibco Data Engineer JD
Role Overview:
We have an immediate need for a senior-level Big data Engineer with core Java skills to join our TIBCO Mashery team. You'll have an opportunity to work on the Mashery Datamart that leverages Big data and Cloud technologies such as Kafka, Spark, Hadoop, AWS, etc. Mashery Datamart is a critical component of the Mashery API Management product that provides data to customers that can be used for performance and usage measurements and for billing functions.
Responsibilities:
Creating complex data processing pipelines, using apache spark/hadoop.
Maintain and expand batch aggregations/reports to existing data mart processing using Hive/Oozie.
Maintain and expand real-time real-time aggregations using Spark streaming,Cassandra,Graphite, Elasticsearch.
Writing Unit tests, integration tests and help in defining and executing performance Tests for Datamart components.
Deploying data pipelines in production based on Continuous Delivery practices.
Create Dashboards/Alerts exhibiting critical metrics and KPI (key performance indicators) for monitoring/tuning and scaling data processing pipelines.
Work with Senior Architects to define next generation Architecture for Datamart.
Work Closely with Operations team to size, scale and tune existing and new infrastructure.
Clearly, communicate ideas, thought process, different alternatives for design and architecture in Discussions and written communication (wiki/design documents/internal blogs).
Qualifications:
7- 10+ years’ experience with a BSCS, 5-7+ years with a MSCS
3+ years of experience building and deploying large scale data processing pipelines in a production environment.
Good to have experience on Cloudera Hadoop.
Production-level hands-on experience working on HDFS, Java MapReduce, Hive, Apache Spark, Oozie and other tools in Big data stack.
Hands-on experience with one or more NOSQL databases like - Cassandra, Elasticsearch, Redis.
Knowledge of software best practices, like Test-Driven Development (TDD) and Continuous Integration (CI)
Solid knowledge of multi-threaded design approach, concurrency and Distributed Systems.
Good to have experience in working on SaaS (Software as a service) product in cloud
Knowledge of software development methodologies including agile, TDD, CI/CD
Knowledgeable of Unix scripting language such as Bash, Perl, PHP, Python
You'll enjoy working in a fast-paced environment with growth potential.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment