0

I know in PHP or Java you can do something like this

int usefulCharacters = text == null ? 0 : text.length();

So in Python i want to check if after using the filter(item,iterable) function and return true if item is appeared in the iterable:

Note: valid_indices is in rdd format so i have to convert it into a list so it will be iterable

is_index1 = (filter(index_md5,valid_indices.collect()) ? true:false)

Seems like Python doesn't like the "?" syntax though. Please advise, i'm learning Python and thank you!

juanpa.arrivillaga
  • 65,257
  • 7
  • 88
  • 122
Alvis Luu
  • 15
  • 4

2 Answers2

1

In python, the ternary operator is using syntax : True if <condition> else False

so, Following your code sample, it will give:

is_index1 = True if filter(index_md5,valid_indices.collect()) else False

Note that I also use the True and False values to use Python language booleans.

jphautin
  • 36
  • 2
  • the syntax is good but i need to tweak a little bit inside of the if condition to make it works properly. Thank you anyways! – Alvis Luu May 03 '20 at 22:44
1

You are probably looking for this (df being PySpark data frame, and not RDD)

from pyspark.sql import functions as F
from pyspark.sql.functions import col, size

Res=df.select(F.when(F.col('text').isNull(), lit(0)).otherwise(F.size(F.col('text'))).alias('new_col'), <other columns>)
Grzegorz Skibinski
  • 11,497
  • 2
  • 7
  • 30