Today i will teach you , how you generate dates YYYY-MM-DD format dynamically if you have given month and year. This will generate response in the mentioned format with the given values.
# Import required modules
from pyspark.sql import SparkSession
from pyspark.sql.functions import col, lit, sequence, last_day, date_format, concat_ws
# Set input values
ep_month = 5
ep_year = 2023
# Create SparkSession
spark = SparkSession.builder.appName("DaysOfMonth").getOrCreate()
# Create DataFrame with one row and two columns: ep_month and ep_year
df = spark.createDataFrame([(ep_month, ep_year)], ["ep_month", "ep_year"])
# Create a sequence of days in the given month using the last_day function
last_day_str = concat_ws("-", col("ep_year"), col("ep_month"), lit("01"))
last_day_of_month = last_day(last_day_str, "dd")
days_seq = sequence(lit(1), last_day_of_month)
# Use date_format and concat_ws functions to format each day in the yyyy-mm-dd format
formatted_dates = df.select(date_format(concat_ws("-", col("ep_year"), col("ep_month"), col("day")), "yyyy-MM-dd").alias("date"))
# Retrieve and print each day in the month
for row in formatted_dates.collect():
print(row.date)
Yeah!! Now you know how to code this type of questions if an interviewer asked during the interview.
Thanks, if you learn something today, please do comment, share and like my pages.
0 Comments