Stack Overflow
Stack Overflow

Questions tagged [spark-cassandra-connector]

Connects Apache Spark and Cassandra for clustered queries

Summary

Get lightning-fast cluster computing with Spark and Cassandra. This library lets you expose Cassandra tables as Spark RDDs, write Spark RDDs to Cassandra tables, and execute arbitrary CQL queries in your Spark applications.

Links

839 questions
0
votes
1 answer

Scala wrapping method of a parametrized class (spark-cassandra-connector)

I am writing a set of methods that extend Spark RDD's API. I have to implement a general method for storing the RDDs, and for a start I tried to wrap spark-cassandra-connector's saveAsCassandraTable, without success. Here's the "extending RDD's API"…
bmscicho
  • 109
  • 1
  • 11
0
votes
1 answer

How to convert nested case class into UDTValue type

I'm struggling using custom case classes to write to Cassandra (2.1.6) using Spark (1.4.0). So far, I've tried this by using the DataStax spark-cassandra-connector 1.4.0-M1 and the following case classes: case class Event(event_id: String,…
Dennis Hunziker
  • 1,263
  • 9
  • 18
0
votes
1 answer

Error when running job that queries against Cassandra via Spark SQL through Spark Jobserver

So I'm trying to run job that simply runs a query against cassandra using spark-sql, the job is submitted fine and the job starts fine. This code works when it is not being run through spark jobserver (when simply using spark submit). Could someone…
vicg
  • 1,037
  • 11
  • 30
0
votes
1 answer

Dataframe is not saved into Cassandra

I have one application with Spark (version 1.4.0) and Spark-Cassandra-connector (version 1.3.0-M1). In which, I am trying to store one dataframe into Cassandra table which has two columns (total, message). And i already created table into Cassandra…
Kaushal
  • 2,981
  • 3
  • 27
  • 44
0
votes
1 answer

Error running spark app using spark-cassandra connector

I have written a basic spark app that reads and writes to Cassandra following this guide (https://github.com/datastax/spark-cassandra-connector/blob/master/doc/0_quick_start.md) This is what the .sbt for this app looks like: name := "test…
user3376961
  • 747
  • 1
  • 10
  • 17
0
votes
1 answer

Spark Cassandra SQL can't perform DataFrame methods on query results

So I have a Spark-Cassandra cluster that I am trying to execute sql queries on. I build a jar with sbt assembly then I submit it with spark-submit. This works fine when I am not using spark-sql. When I am using spark sql I get an error, below is the…
vicg
  • 1,037
  • 11
  • 30
0
votes
1 answer

How can I customize column-mapping when save RDD to Cassandra?

I am using Java to write Spark application. If I have a customized tuple, let's say class "Person". Class Person { public String name1; public String name2; public String name3; } And I have a JavaRDD rdd; Now I want to save it…
Jintao Guan
  • 101
  • 1
  • 10
-1
votes
1 answer

Apache Spark : Cassandra Read : How to Ignore or replace '\n' character while writing to a file

I am having some column entries(string) in my cassandra which is having '\n' inside like: (id,name,age) values(1,'abc\nxyz',28) Now I am using spark to write my rows in a csv file but spark is taking '\n' character as a new line val cass=…
Vicky
  • 883
  • 13
  • 28
-1
votes
1 answer

How to load csv couple of lines per couple of lines

I'm connecting Spark to Cassandra and I was able to print the lines of my CSV using the conventional COPY method. However, if the CSV was very large as it usually happens in Big Data, how could one load the CSV file couple of lines per couple of…
Amanda Kerr
  • 1
  • 4
-1
votes
2 answers

Can not write spark Dataframe into cassandra table

I am connecting spark on HDP3.0 with Cassandra to write a dataframe into cassandra table but receiving below error: enter image description here enter image description here my code to write into cassandra table is below: enter image description…
Bensassi
  • 1
-1
votes
1 answer

How to convert tabular format data to sentence or readable format using Pyspark?

Here is the table format in image so how should i convert it into readable format like it should display as- member_id is belongs to region and so on for other columns So, can anyone help me out how to write a function to convert tabular format…
Instinct
  • 31
  • 5
-1
votes
1 answer

Failed to launch spark-shell with spark-cassandra-connector find the log

$SPARK_HOME/bin/spark-shell --packages datastax:spark-cassandra-connector:2.0.0-M2-s_2.11 --conf spark.cassandra.connection.host=127.0.0.1 Ivy Default Cache set to: /home/gopalkrishna/.ivy2/cache The jars for the packages stored in:…
Venu Gopal
  • 1
  • 2
-1
votes
2 answers

Parse Spark RDD after Cassandra join

I want to read a parquet file and join with some of the columns in Cassandra. I was able to join and get an RDD, but I am unable to parse the resulting RDD. Here are further details case class IP (key: String, key2: String,key3: String,key4:…
CSUNNY
  • 153
  • 12
-1
votes
1 answer

Which versions should be used for Spark cassandra connection using java Language?

I'm trying to connect spark and cassandra database using Java language. For connecting spark and cassandra I'm using latest version of Spark-cassandra-Connector i.e 2.4.0. Currently I can connect spark and cassandra using connector. I am getting…
Kavita Mahankal
  • 11
  • 3
-1
votes
1 answer

What is LongAdder related to cassandra+spark connector?

When i load data into cassandra with using databricks, its getting the issue with Caused by: java.lang.NoClassDefFoundError: com/twitter/jsr166e/LongAdder Its simple saveToCassandra to table. I looked this twitter jsr166e jar in maven , its very…
Nathon
  • 129
  • 3
  • 12
1 2 3
…
55
56