Enabling data ingestion with various data sources for routing, processing data from sources to Amazon S3, HDFS, NoSQL and Apache Kafka