vignesh asokan
vignesh asokan

Reputation: 145

How to get week number using year and day of year using pyspark?

I am trying to add row numbers to a table. I need to add 1 for the first 7 rows in the dataframe and then 2 for the second 7 rows in the dataframe and so on. for eg pls refer to the last column in the dataframe. I am basically trying to get week number based on day of the year and year

  +-----------+---------------+----------------+------------------+---------+
  |datekey    |datecalendarday|datecalendaryear|weeknumberofseason|indicator| weeknumber
  +-----------+---------------+----------------+------------------+---------+
  |4965       |1              |2018            |2                 |1        |  1
  |4966       |2              |2018            |2                 |2        |  1
  |4967       |3              |2018            |2                 |3        |  1
  |4968       |4              |2018            |2                 |4        |  1
  |4969       |5              |2018            |2                 |5        |  1
  |4970       |6              |2018            |2                 |6        |  1
  |4971       |7              |2018            |3                 |7        |  1
  |4972       |8              |2018            |3                 |8        |  2
  |4973       |9              |2018            |3                 |9        |  2
  |4974       |10             |2018            |3                 |10       |  2
  |4975       |11             |2018            |3                 |11       |  2
  |4976       |12             |2018            |3                 |12       |  2
  |4977       |13             |2018            |3                 |13       |  2
  |4978       |14             |2018            |4                 |14       |  2

Upvotes: 0

Views: 152

Answers (1)

vignesh asokan
vignesh asokan

Reputation: 145

I stumbled upon a solution where i use ntile function to get the number of week from the days available in that year. Any other effecient solution also would help. Thaks in advance

Upvotes: 0

Related Questions