Use amazon-data-pipeline tag instead
Simple service to transfer data between Amazon data storage services, kick off Elastic MapReduce jobs, and connect with outside data services.
Questions tagged [aws-data-pipeline]
67 questions
0
votes
1 answer
AWS Copy data from one database to an otherone
so here's the thing, we (on the company I work) have a production database (MySQL) and we want to create a new database (PostgreSQL) for BI/Analytics.
The whole infrastructure is on AWS, and we have to copy the data from the production to the…
Juan
- 75
- 11
0
votes
1 answer
Associate Elastic IP with scheduled AWS data pipeline
Anybody knows is it possible to associate Elastic IP with scheduled data pipeline? I have configured data pipeline to run every day. During data pipeline execution, I need access to Google DB. To have access to Google DB I should add IP (CIDR) in DB…
J2James
- 23
- 4
0
votes
1 answer
AWS Datapipeline incorrect java version
I am trying to execute a jar file in my datapipeline and it is erroring out in a fashion that indicates to me that the version of java that is installed in my pipeline is lower than that required by the executable jar. I have tried to add a command…
Matt
- 11
0
votes
1 answer
Alarm when object size in S3 bucket exceeds threshold
I have AWS data pipelines setup that feed to my S3 bucket. Each time a new feed file is generated by the pipeline and stored in the bucket. We keep at most 30 days of data in the bucket. Is it possible to configure an alarm so that I am notified via…
Aritra Kundu
- 75
- 5
0
votes
1 answer
Default datetime with AWS Data Pipeline
We are using data pipeline to push the data from s3 to the amazon redshift table. We are pushing the data using JSON
{
"jsonpaths": [
"$['data1']",
"$['data1']",
"$['data1']['data1']",
"$['data1']['data1']",
…
user3681970
- 993
- 3
- 15
- 33
0
votes
1 answer
Load csv file from S3 to RDS Mysql using AWS data pipeline
I'm new to AWS, trying to load data from csv file to RDS MySQL using data pipeline "load S3 data into RDS mysql table" template. But getting error "DriverClass not found for database:aurora". Tried setting JDBC uri property point to driver jar in…
acs
- 11
- 2
-1
votes
1 answer
Invoke Lambda function from AWS DataPipeline
Is there a way to invoke a Lambda function from AWS DataPipeline? Looking at the activities available, there isn't a direct way to invoke a lambda from DataPipeline.
