Solving PowerQuery Puzzles with R

by jsendak | Apr 3, 2024 | DS Articles

#169–170

Puzzles

Author: ExcelBI

All files (xlsx with puzzle and R with solution) for each and every puzzle are available on my Github. Enjoy.

Puzzle #169

In todays challenge we have certain pattern to extract from given texts. As you may notice I really like string manipulation tasks and I love to make them solved using Regular Expresions. In this task we need to take those which: starts with capital letter, does not contain lower case letters, contains capital letters AND digits. Pretty interesting conditions, so lets find out the way.

Loading data and library

library(tidyverse)
library(readxl)

input = read_excel("Power Query/PQ_Challenge_169.xlsx", range = "A1:A8")
test = read_excel("Power Query/PQ_Challenge_169.xlsx", range = "C1:D8")

Transformation

pattern = ("b[A-Z](?=[A-Z0-9]*[0-9])[A-Z0-9]*b")

result = input %>%
  mutate(Codes = map_chr(String, ~str_extract_all(., pattern) %>% unlist() %>%
                              str_c(collapse = ", "))) %>%
  mutate(Codes = if_else(Codes == "", NA_character_, Codes)) 

My Regexp can look weird so let me break the mystery down:

• b stands for word boundaries, we want to extract parts of text that do not have whitespaces or punctuation. Even if they are not linguistical words :).
• [A-Z] at the beginning means that first character must be a letter (between A and Z). If we would have to analyse whole text with spaces and everything we would use ^ to emphasize beginning of string, not word.
• At the end we have [A-Z0–9]* that stands for zero or more digits OR capital letters.
• Last part placed in the middle is called positive lookahead. (?=[A-Z0–9]*[0–9]) make us sure that these zero or more characters from previous point, where preceded with optional letters or digits [A-Z0–9]*and mandatory digits [0–9] . All together means that our string has at least capital letter at the beginning and one digit at the end.

Validation

all.equal(test$Codes, result$Codes)
# [1] TRUE

Puzzle #170

Power Query Challenges are usually about transforming data from one form to another, sometimes only about the structure, sometimes data are transformed as well. This challenge is sales summary. From over 90 purchases we need to summarise total revenue for weekdays and weekends, and point the most and the less popular product within those transactions. Check it up.

Load libraries and data

library(tidyverse)
library(readxl)

input = read_excel("Power Query/PQ_Challenge_170.xlsx", range = "A1:C92")
test  = read_excel("Power Query/PQ_Challenge_170.xlsx", range = "E1:H3")

Transformation

result = input %>%
  mutate(week_part = ifelse(wday(Date) %in% c(1, 7), "Weekend", "Weekday")) %>%
  summarise(total = sum(Sale),
            .by = c(week_part, Item)) %>%
  mutate(min = min(total),
         max = max(total),
         full_total = sum(total),
         .by = c(week_part)) %>%
  filter(total == min | total == max) %>%
  mutate(min_max = ifelse(total == min, "min", "max")) %>%
  select(-c(total, min, max)) %>%
  pivot_wider(names_from = min_max, values_from = Item, values_fn = list(Item = list)) %>%
  mutate(min = map_chr(min, ~paste(.x, collapse = ", ")),
         max = map_chr(max, ~paste(.x, collapse = ", ")))

colnames(result) <- colnames(test)

Validation

identical(result, test)
# [1] TRUE

Feel free to comment, share and contact me with advices, questions and your ideas how to improve anything. Contact me on Linkedin if you wish as well.

PowerQuery Puzzle solved with R was originally published in Numbers around us on Medium, where people are continuing the conversation by highlighting and responding to this story.

Continue reading: PowerQuery Puzzle solved with R

What the original text covers

The author showcases how to solve two different puzzles (#169 and #170) using R programming. The solutions for both puzzles are available on Github. Both puzzles require data manipulation using libraries such as tidyverse and readxl.

Puzzle #169 involves string manipulation to extract parts of text that meet certain conditions: they start with a capital letter, they don’t contain lower case letters, and they contain capital letters and digits. The author goes into detail about how Regular Expressions (Regexp) are used to achieve this.

Puzzle #170 is about data transformation and summarization. This challenge requires a sales data set with over 90 purchases to be summarized in terms of total revenue for weekdays and weekends, and to identify the most and least popular product within those transactions. The author again uses the tidyverse and readxl libraries to perform these data transformations.

Implications

The solutions to these puzzles demonstrate the efficiency of R programming in solving complex data analysis tasks. They are a reflection of the growing shift towards automated data transformation and the application of advanced algorithms for data extraction.

Given the constant advancements in technology and data science, data manipulation tasks like these are only expected to become more intricate and complex. This means that data scientists must continually refine their skills and stay updated with the latest trends in the field.

Possible Future Developments

In the future, we can expect even more sophisticated libraries and functions in R that can handle complex data manipulation tasks. These developments might include a wider range of data structures to work with, enhanced data visualization capabilities, and more advanced machine learning algorithms.

Actionable Advice

For those interested in data analysis, particularly using R, it is beneficial to constantly challenge yourself with puzzles like these. Practice is essential in mastering the art of data preprocessing and transformation. Additionally, staying updated with the latest data manipulation libraries will be a great boost to your data analysis skills.

For other people using your data, providing clear documentation of your analysis process (as the author does in this case) is highly recommended. This would make your work more accessible and easier to understand by others who might need to use or learn from your data manipulations.

As a data analyst or scientist, always remember to validate your results. As demonstrated in the text, you should always ensure that your final output matches your expectation. This would help prevent any errors or misunderstandings in your analysis results.

Read the original article