Data transformation is the process of converting data from one format or structure into another format or structure. This can range from a simple transformation like transforming a comma-separated list to a line-break-separated list to complex transformations like speech-to-text. Strategies and technologies used can vary widely based on the complexity, volume, format and structure of the data being transformed.
Questions tagged [data-transform]
152 questions
16
votes
3 answers
Rename Azure Storage Table?
Is it not possible to rename an Azure Storage Table?
I cannot seem to find anything online (not even cmdlets). There are no options for this in Visual Studio Server Explorer, Cloud Storage Studio or TableXplorer.
Dave New
- 34,265
- 48
- 183
- 366
3
votes
2 answers
Use dplyr's _if() functions like mutate_if() with a negative predicate function
According to the documentation of the dplyr package:
# The _if() variants apply a predicate function (a function that
# returns TRUE or FALSE) to determine the relevant subset of
# columns.
# mutate_if() is particularly useful for transforming…
MS Berends
- 2,677
- 1
- 21
- 35
3
votes
6 answers
Remove duplicates from array of objects but keep one property as an array
I have a collection like this:
const data = [
{index: 1, number: 's1', uniqId: '123', city: 'LA'},
{index: 2, number: 's2', uniqId: '321', city: 'NY'},
{index: 3, number: 's3', uniqId: '123', city: 'LA'},
{index: 4, number: 's4', uniqId: '111',…
tumbleweeder
- 31
- 4
2
votes
1 answer
Quickly remove tuples that contain tuples of other list
I would like to remove all the tuples in List A that contain a tuple in List B.
This is normally a trivial matter, but I have 10 million records in List A and 200K in List B. My current script (see below) is very slow (~10 seconds for each scan of…
FirstRedPepper
- 412
- 5
- 18
2
votes
2 answers
Converting columns with date to rows in R
Let's say we have a data.frame in R like this:
d = data.frame('2019q1' = 1, '2019q2' =2, '2019q3' = 3)
Which looks like this:
X2019q1 X2019q2 X2019q3
1 1 2 3
How can I transform it to looks like this:
Year Quarter …
Mehdi Zare
- 779
- 1
- 7
- 22
2
votes
1 answer
Exponential distribution of values between a given range in Python
I have three variables Min=0.29, Max=6.52 and center = 2.10. I wish to create a table that distributes this data into 100 values in a table format in the following fashion:
Here, this image can be split into two parts 0 to 50 and 50 to 100.
In the…
Abhishek
- 846
- 2
- 8
- 22
2
votes
1 answer
Filtering data values in one column based on another column and then inserting values into different columns in same SQL Table
This is a bit of a conundrum I am trying to solve using SSIS and a conditional-split transformation. I have a .csv file that contains attribute data in one row for each unique user and the values for each attribute in another column.…
ManMadeNova
- 71
- 12
2
votes
0 answers
Transforming rows to columns using Amazon Glue
I'm using AWS Glue to transform a set of data which have multiple rows based on one column's value. I am trying to pivot the data and reduce the number of rows.
I believe I can use Amazon Glue for this. But, this is my first time using Glue. I'm…
yaja6021
- 121
- 1
- 12
2
votes
1 answer
Dividing the values by their mean for many variables
I wish to conduct a data Transformation by dividing each case in a variable by that variable's mean. I have 91 variables in my dataset. I create the means using the AGGREGATE function:
AGGREGATE
/OUTFILE=* MODE = ADDVARIABLES
/BREAK=
/mean_1…
user10579790
- 325
- 1
- 9
2
votes
3 answers
Converting character matrix with vectors of numerics and plain numerics into numeric
this in theory simple task turned out to drive me crazy today. I'm rather new to R, but got along quite well until now. Maybe someone of you is having an easier time to solve it.
In short: How do I get the maximum values per observation out of a…
3bbing
- 75
- 5
2
votes
1 answer
Label day timing into morning, afternoon and evening in R
How can i label time of the day (Morning, Afternoon and Evening) for given timestamps?
Initial Data
Id Time_stamp
3083188c 2016-08-29 13:10:51
924d500e 2016-08-29 09:22:33
ad4dd7ff 2016-08-25 20:29:35
Final data
Id …
ajax
- 121
- 10
2
votes
3 answers
Azure Table Storage - remove columns
I think this is not possible, but however I ask the question, maybe I have missed something.
Can we add/remove columns from an azure table?
For example by default we get those columns: PartitionKey, RowKey, Timestamp, ETag. Can I add for example…
user2818430
- 5,195
- 15
- 63
- 133
2
votes
4 answers
How to transform a comma-separated string to line-break-separated string?
I have a comma-separated string. How can I convert it into line-break-separated format. My string looks like this:
red,yellow,green,orange,pink,black,white
And needs to be formatted to in this way:
red
yellow
green
orange
pink
black
white
Here is…
Arun
- 1,332
- 9
- 27
- 55
1
vote
2 answers
Convert array of Objects into a grouped array of Objects Typescript
I'm trying to convert an array of objects like this:
[{grandParentField:'grandParent1', parentField:'parent1', childField: 'child1'},
{grandParentField:'grandParent1', parentField:'parent1', childField: 'child2'},
{grandParentField:'grandParent2',…
DontThinkJustGo
- 190
- 10
1
vote
2 answers
Sequential Increase in Column value based on a condition R
I have an R data frame that has an ID column with multiple records for an ID. When the flag is set to 1 for an ID, I want to create a column new timeline that starts from 1 and increases sequentially in increments of 6 (1,6,12...). How can I achieve…
Datamaniac
- 47
- 1
- 5