Exploring Solar Eclipses: Data Scraping and Mapping

Exploring Solar Eclipses: Data Scraping and Mapping

[This article was first published on r.iresmi.net, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)


Want to share your content on R-bloggers? click here if you have a blog, or here if you don’t.

A photo of solar eclipse

2017 Solar Eclipse with Totality – Composite – CC-BY-NC by Jeff Geerling

I saw this post showing a map of the year of the most recent total eclipse, and people mentioning that we can find the data on the Five Millennium Canon of Solar Eclipses Database (the data also mentioned in the thread on ArcGIS don’t go before the seventeenth century).


There is a bit of involved scraping because manually we can’t get more than 4 eclipses in a go and part of the generation is driven by javascript, but we can eventually get the whole 2538 eclipses between -1999 and 2024!

Setup

library(tidyverse)
library(rvest)
library(glue)
library(httr)
library(sf)
library(mapview)

Data

First we make the query on the site: all total eclipse. It populates a select HTML control where we can manually scrape the eclipses dates with the browser tools. I saved them in a CSV file in the data directory.

We will next chose an area of interest (AOI). I’ll open a spatial data of Metropolitan France (built by merging all regions) ; pick your own, in EPSG:4326…

Note

You can use data from Natural Earth with {rnaturalearth} for example. Add a character field eclipse_date to hold the dates of eclipse.

eclipses_dates <- read_csv("data/eclipses_dates.csv",
                           col_types = "c") |>
  pull(astro_dates)

aoi <- read_sf("~/data/adminexpress/adminexpress_cog_simpl_000_2022.gpkg",
                   layer = "region") |>
  filter(insee_reg > "06") |>
  st_union() |>
  tibble(geom = _) |>
  st_sf() |>
  mutate(eclipse_date = NA_character_, .before = 1)

Download files

Based on a half-hidden URL, we will, slowly, ask for the KMZ to be generated, find its URL, get the file and save it in the results directory. It should take an hour…

get_kmz <- function(eclipse_date) {
  message(glue("Downloading: {eclipse_date}"))
  tryCatch({
    read_html(glue("http://xjubier.free.fr/en/site_pages/solar_eclipses/xSE_GoogleEarth.php?Ecl={eclipse_date}&Acc=2&Umb=1&Lmt=1&Mag=0")) |>
      html_elements("fieldset a") |>
      html_attr("href") |>
      GET(write_disk(glue("results/eclipse_{eclipse_date}.kmz")))
  },
  error = function(e) {
    message("  x Can't download/save")
    print(e)
    return(NULL)
  })
}

eclipses_dates |>
  walk(slowly(get_kmz), .progress = TRUE)

If you are eager to start, get a bundle of all of them from here (110 MB). Uncompress in results.

Prepare the processing

We need to open the right layer in each KMZ, as there are many of them with varying names, then we intersect it with our AOI, recursively. So we make one function for each task…

Warning

Many layers have geometry errors for the S2 engine: we will skip them ; so beware the map may be inaccurate!

get_polygon <- function(eclipse_date) {
  tryCatch({
    layer <- st_layers(glue("results/eclipse_{eclipse_date}.kmz")) |>
      pluck("name") |>
      keep((x) str_detect(x, "Umbral Path"))

    read_sf(glue("results/eclipse_{eclipse_date}.kmz"),
            layer = layer) |>
      st_zm() |>
      st_make_valid() |>
      st_collection_extract("POLYGON") |>
      transmute(eclipse_date = eclipse_date)
  },
  error = function(e) {
    message("  x Error in opening")
    print(e)
    return(NULL)
  })
}

crop_map <- function(current_map, eclipse_date) {
  message(glue("Date: {eclipse_date}"))
  p <- get_polygon(eclipse_date)
  tryCatch({
    if (any(apply(st_intersects(current_map, p), 1, any))) {
      message("  -> Eclipse matches area of interest")
      bind_rows(st_difference(current_map, p) |>
                  select(eclipse_date),
                st_intersection(current_map, p) |>
                  mutate(eclipse_date = eclipse_date.1) |>
                  select(-eclipse_date.1))
    } else {
      current_map
    }
  },
  error = function(e) {
    message("  x Error in intersection")
    print(e)
    return(current_map)
  })
}

And we run them:

aoi_eclipse <- eclipses_dates |>
  reduce(crop_map, .init = aoi) |>
  write_sf("aoi_eclipse.gpkg")

You can use the Geopackage in QGIS, or just display it here:

eclipse_year <- aoi_eclipse |>
  st_make_valid() |>
  group_by(year = str_sub(eclipse_date, 1, 5)) |>
  summarise()

mapview(eclipse_year)

To leave a comment for the author, please follow the link and comment on their blog: r.iresmi.net.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you’re looking to post or find an R/data-science job.


Want to share your content on R-bloggers? click here if you have a blog, or here if you don’t.

Continue reading: Eclipse map

Analysis of Total Solar Eclipses: Leveraging Data and Mapping Techniques

This article demonstrates how a researcher used various data and mapping strategies to visualise the path of total solar eclipses from years -1999 to 2024. The researcher has used an informative data source known as the Five Millennium Canon of Solar Eclipses Database, and used the programming language R, along with libraries such as tidyverse, rvest, glue, httr, sf, mapview, etc. to process this data.

Long-term Implications

This exercise has long-term implications for not just astrologists, but also for data researchers, environmentalists, educators, programmers, and common public as well. By mapping the path of solar eclipses, one can predict the locations where they can witness this rare phenomenon in future. This can aid the efforts of scientists to gather more data about solar eclipses, and astronomers to forecast future astronomical events with better precision.

Possible Future Developments

In future, this technique can be refined and adapted to other similar phenomena like lunar eclipses or transits of planets. It may also be possible to use more sophisticated data science techniques to extract deeper insights from the data. Some of these might include machine learning algorithms to predict the next locations where solar eclipses will be visible, or the use of more interactive data visualization tools to make the predictions more user-friendly and accessible.

Actionable Advice

Here is some advice on how to leverage these insights:

  • For Researchers and Scientists: Utilize the Five Millennium Canon of Solar Eclipses Database and programming languages like R to forecast and analyze solar events, for advancing scientific knowledge.
  • For Educators: Use these visualizations to create immersive and practical learning experiences for students studying astronomy.
  • For Data Analysts/Scientists: Implement and expand on this technique for other kinds of environmental or spatial data to help drive field-specific innovations.
  • For General Public: Use these maps to find out when and where you can observe the next solar eclipse, to enhance your knowledge and fulfill your curiosity about astronomical events.

Read the original article

Crafting Elegant Scientific Documents: LaTeX and R Markdown Tutorial

Crafting Elegant Scientific Documents: LaTeX and R Markdown Tutorial

[This article was first published on Numbers around us – Medium, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)


Want to share your content on R-bloggers? click here if you have a blog, or here if you don’t.

Introduction

In the world of scientific research and academic writing, the clarity, precision, and aesthetics of your documents can significantly impact their reception and comprehension. LaTeX, a powerful typesetting system, has long been revered for its ability to create beautifully formatted documents, especially those requiring complex mathematical expressions and detailed layouts. However, the steep learning curve associated with LaTeX can deter many. Enter R Markdown, a tool that simplifies the creation of dynamic documents, presentations, and reports directly from R code. When combined with the versatility of RStudio, it offers a more accessible entry point into the world of LaTeX, without sacrificing the depth and precision that professional documents require.

This tutorial aims to bridge the gap between the high-quality typesetting capabilities of LaTeX and the dynamic, code-integrated documentation of R Markdown. Whether you’re compiling research findings, drafting an academic paper, or preparing a report with rich data visualizations, integrating LaTeX with R Markdown in RStudio enhances both the appearance and functionality of your work. By the end of this guide, you’ll be equipped with the knowledge to leverage the best of both worlds, crafting documents that stand out for their elegance and precision.

Prerequisites and Setup

Installing RStudio and LaTeX

Before we dive into the intricacies of combining LaTeX with R Markdown, let’s ensure you have all the necessary tools installed. RStudio is an indispensable IDE for anyone working with R, and it provides seamless support for R Markdown. LaTeX, on the other hand, is a typesetting system that excels in document preparation, especially for those containing complex mathematical formulas.

  • RStudio: If you haven’t already, download and install RStudio. Choose the version appropriate for your operating system.
  • LaTeX Distribution: For LaTeX, you need a distribution based on your operating system. Windows users can opt for MiKTeX, macOS users for MacTeX, and Linux users for TeX Live. Installation links and instructions are readily available on their respective websites.

After installing both RStudio and your LaTeX distribution, ensure that RStudio can locate your LaTeX installation. This integration is typically automatic, but you can verify or adjust the settings in RStudio by navigating to Tools > Global Options > Sweave.

Configuring RStudio for LaTeX and R Markdown

With RStudio and LaTeX installed, the next step is to configure your RStudio environment for an optimal working experience. This involves:

  • Installing Necessary R Packages: Open RStudio and install the rmarkdown package, which supports the integration of R code with Markdown (and by extension, LaTeX) for dynamic document generation. Install it by running:
install.packages("rmarkdown")
  • Testing Your Setup: To confirm everything is set up correctly, create a new R Markdown document. Go to File > New File > R Markdown… , then choose PDF as the output format. This action requires LaTeX for PDF generation, so if it succeeds without errors, your setup is correct.

This section’s goal is to ensure you have a smooth start with all the necessary tools at your disposal. Once you’re set up, the real fun begins: exploring the synergy between LaTeX and R Markdown to create stunning scientific documents.

Your First R Markdown Document with LaTeX

Creating your first R Markdown document integrated with LaTeX in RStudio is a simple yet exciting process. This section will guide you through creating a basic document, adding LaTeX for formatting and equations, and generating a PDF output.

Creating an R Markdown Document

  1. Start a New R Markdown File: In RStudio, go to File > New File > R Markdown… This opens a dialog where you can set the document’s title and output format. For now, select PDF and click OK.
  2. Explore the Default Content: RStudio will generate a sample document filled with some basic Markdown content and example code chunks. This template serves as an excellent introduction to R Markdown’s capabilities.

Integrating Basic LaTeX Elements

Within your R Markdown document, you can start integrating LaTeX directly. Here’s how you can add some basic LaTeX commands for text formatting and sections:

This is an R Markdown document with LaTeX. Markdown allows you to write using an easy-to-read, easy-to-write plain text format, which then converts to LaTeX for high-quality document production.

section{Introduction}
This is a section created using LaTeX.

subsection{Background}
This subsection provides background information, also formatted using LaTeX.

textbf{Bold text} and textit{italicized text} can easily be added with LaTeX commands.

Adding Mathematical Expressions

One of LaTeX’s strengths is its ability to format complex mathematical expressions beautifully. In R Markdown, you can include these expressions by enclosing them in dollar signs for inline equations or double dollar signs for displayed equations:

Here is an inline equation: (E=mc^2).

And a displayed equation:

$$
a^2 + b^2 = c^2
$$

Compiling to PDF

After adding your content, compile the document to PDF by clicking the “Knit” button in RStudio and selecting PDF. RStudio will use LaTeX to process your document, incorporating any LaTeX commands or mathematical expressions you’ve included, and generate a PDF.

This simple exercise demonstrates the power of combining R Markdown’s dynamic capabilities with LaTeX’s typesetting prowess, all within the RStudio environment. Whether you’re documenting research findings, drafting a paper, or preparing a report, this approach allows you to create professional, elegantly formatted documents efficiently.

Advanced LaTeX Features in R Markdown

Having grasped the basics of integrating LaTeX into R Markdown documents, we’ll now delve into advanced features to further elevate your scientific document’s quality. This segment highlights enhanced figure and table management, utilizing custom LaTeX commands, and effectively handling bibliographies within RStudio.

Working with Figures and Tables

LaTeX is renowned for its precise control over figures and tables, but in R Markdown, we approach these elements differently, leveraging Markdown and R code chunks for dynamic content integration and formatting.

Figures

For static images, use Markdown syntax:

![Caption for the figure.](my_address_to_logo){width=20%}

For dynamically generated figures from R:

```{r label, echo=FALSE, fig.cap="Caption for the figure."}
data(mtcars)
plot(mtcars$wt, mtcars$mpg)
```

Tables

To create detailed and customizable tables in your R Markdown document using LaTeX, you’ll directly use the tabular environment provided by LaTeX. This allows for precise control over the table's appearance, alignment, and overall structure. Here's a basic example of creating a table with LaTeX:

begin{table}[h]
centering
caption{Sample Data Table}
begin{tabular}{lcr}
hline
textbf{Left Align} & textbf{Center} & textbf{Right Align} 
hline
Data 1 & Data 2 & Data 3 
More & Data & Here 
hline
end{tabular}
label{tab:sample_table}
end{table}

This LaTeX code snippet places a table with headers aligned to the left, center, and right. The hline command creates horizontal lines for clarity, and textbf is used for bold header text. The caption{} and label{} commands are used for the table's caption and referencing it in the text, respectively.

Defining and Using Custom LaTeX Commands

You can define custom LaTeX commands for repetitive tasks or to simplify complex formatting. Custom commands are defined in the YAML header of your R Markdown document using header-includes:

header-includes:
  - newcommand{highlight}[1]{textbf{textcolor{red}{#1}}}

This command, highlight{}, makes specified text bold and red. To use this command within your document:

This is regular text and this is highlight{highlighted text}.

Applying Custom Commands in Tables

Your custom LaTeX commands can be utilized within tables to emphasize specific pieces of data or apply consistent formatting. Using the previously defined highlight{} command:

begin{table}[h]
centering
caption{Demonstrating Custom Commands in Tables}
begin{tabular}{lc}
hline
textbf{Description} & textbf{Data} 
hline
Regular Data & 123 
Highlighted Data & highlight{456} 
hline
end{tabular}
label{tab:custom_command_table}
end{table}

This example shows how to apply the highlight{} command within a table to make specific data stand out.

In this chapter, we’ve explored how to enhance your R Markdown documents with figures and sophisticated table formatting using LaTeX and the creation and application of custom LaTeX commands. Starting with the tabular environment, we demonstrated the method to craft detailed tables that meet specific aesthetic and structural requirements. Additionally, we covered how to define and utilize custom LaTeX commands within your document, allowing for efficient and consistent formatting across your scientific documents. This approach ensures that your work not only conveys information effectively but also adheres to the high standards of professional and academic presentation.

Crafting Complex Scientific Equations with LaTeX in R Markdown

The seamless integration of LaTeX within R Markdown particularly shines when dealing with complex scientific equations, which are cumbersome, if not impossible, to accurately represent in plain text or basic Markdown. LaTeX provides a comprehensive set of tools for typesetting mathematical expressions, from simple fractions to elaborate equations used in advanced physics and mathematics. This chapter demonstrates how to leverage LaTeX for this purpose within an R Markdown document.

Basic Mathematical Expressions

LaTeX allows for the inline and block display of mathematical expressions. For inline equations, enclose your LaTeX code in single dollar signs ($), and for equations that should be displayed as a separate block, use double dollar signs ($$).

Inline Equation:

Einstein's famous equation can be represented inline as $E=mc^2$.

Displayed Equation:

$$E=mc^2$$

This displays the equation centered on its own line, making it stand out for emphasis.

Advanced Equation Formatting

LaTeX excels in formatting complex equations, such as systems of equations, matrices, and functions involving sums, integrals, and limits.

System of Equations:

$$
begin{align*}
x + y &= 10 
2x - y &= 4
end{align*}
$$

Matrix:

$$
begin{pmatrix}
a & b 
c & d
end{pmatrix}
$$

Integral:

$$
int_0^infty e^{-x}dx
$$

These examples demonstrate just a fraction of the capabilities LaTeX offers for mathematical typesetting. When utilized within R Markdown, it enables authors to seamlessly integrate complex mathematical content into their documents, enhancing both readability and professionalism.

Utilizing LaTeX for Scientific Notation

Scientific documents often require notation that is difficult or awkward to express in other formats. LaTeX addresses this with a broad array of symbols and structures designed specifically for scientific writing:

$$
gamma + pi approx 3.14 text{, where } gamma text{ is the Euler-Mascheroni constant, and } pi text{ is the mathematical constant pi.}
$$

The combination of R Markdown and LaTeX provides a powerful toolset for scientists, mathematicians, and anyone else working with complex equations or scientific notation. It brings together the best of both worlds: the dynamism and reproducibility of R Markdown with the precise typesetting and extensive capabilities of LaTeX.

Some more complex equations

Fourier Series:

$$
f(x) = a_0 + sum_{n=1}^{infty} left( a_n cos frac{2pi nx}{P} + b_n sin frac{2pi nx}{P} right)
$$

Schrodinger equation:

$$
ihbarfrac{partial}{partial t}Psi(mathbf{r}, t) = left[ frac{-hbar^2}{2mu}nabla^2 + V(mathbf{r}, t) right] Psi(mathbf{r}, t)
$$

General relativity field equation:

$$
G_{munu} + Lambda g_{munu} = frac{8pi G}{c^4} T_{munu}
$$

Navier-Stokes Equations for Fluid Dynamics:

$$
rho left( frac{partial mathbf{v}}{partial t} + mathbf{v} cdot nabla mathbf{v} right) = -nabla p + mu nabla^2 mathbf{v} + mathbf{f}
$$

And render of all equations included in chapter.

Compiling Documents and Customizing Outputs in R Markdown

R Markdown provides a seamless workflow for creating dynamic documents, reports, presentations, and more, directly from R. When incorporating LaTeX, you gain additional control over the document’s appearance, enabling the creation of professional-grade scientific documents. This chapter explores how to compile your R Markdown documents into PDFs, leveraging LaTeX for advanced formatting, and how to customize these outputs to fit various academic and professional standards.

Compiling R Markdown Documents to PDF

To compile an R Markdown document to PDF with LaTeX formatting:

  1. Ensure LaTeX is Installed: Before compiling, make sure you have a LaTeX distribution installed on your computer, as discussed in the setup chapter.
  2. Use the ‘Knit’ Button: In RStudio, the simplest way to compile your document is by using the Knit button. When you click Knit, RStudio automatically renders your document into a PDF, incorporating any LaTeX code or styling you’ve included.
  3. Customizing the Build Process: For more control over the compilation process, you can use the rmarkdown::render() function in the R console:
rmarkdown::render("your_document.Rmd", output_format = "pdf_document")

This function allows for additional arguments and customization, offering more flexibility than the Knit button.

Customizing PDF Output with LaTeX

LaTeX allows for extensive customization of PDF output through the use of packages and settings defined in the preamble of your R Markdown document. Here are a few ways to customize your PDF documents:

  • Page Layout and Fonts: Use LaTeX packages such as geometry to adjust margins, fancyhdr for custom headers and footers, and fontspec for font customization.
header-includes:
  - usepackage{geometry}
  - geometry{left=3cm,right=3cm,top=2cm,bottom=2cm}
  - usepackage{fancyhdr}
  - pagestyle{fancy}
  - usepackage{fontspec}
  - setmainfont{Times New Roman}
  • Section Formatting: Customize section titles using the titlesec package.
header-includes:
  - usepackage{titlesec}
  - titleformat*{section}{Largebfseries}
  • Including External LaTeX Files: For complex documents, you might want to maintain your LaTeX preamble in a separate .tex file and include it in your R Markdown document.
header-includes:
  - input{preamble.tex}

Advanced Document Features

Leveraging LaTeX within R Markdown also allows for the inclusion of advanced document features that are typically challenging to implement, such as conditional text rendering, custom automatic numbering for figures and tables, and intricate mathematical typesetting, which we’ve covered in the previous chapter.

The combination of R Markdown and LaTeX offers unparalleled flexibility and power for scientific document creation. By mastering the compilation process and customizing the output, you can produce documents that not only meet the rigorous standards of academic and professional communication but also reflect your personal style and preferences.

Further Resources for Mastering LaTeX in R Markdown

Having explored the fundamentals and some advanced techniques for integrating LaTeX into R Markdown documents, it’s beneficial to know where to look for further information, tutorials, and community support to continue enhancing your skills. This final chapter provides a curated list of resources, including books, online tutorials, forums, and packages, designed to deepen your understanding and proficiency in using LaTeX with R Markdown for creating professional and sophisticated documents.

Books

  1. R Markdown: The Definitive Guide” by Yihui Xie, J.J. Allaire, and Garrett Grolemund. This comprehensive guide provides a thorough introduction to R Markdown, including its integration with LaTeX for producing high-quality documents.
  2. The LaTeX Companion” by Frank Mittelbach and Michel Goossens. A detailed reference book for LaTeX users, covering a wide range of topics from basic document formatting to more complex customizations and extensions.
  3. Practical R Markdown” by Benjamin Soltoff. This book focuses on the practical aspects of using R Markdown in research and data analysis, with sections dedicated to integrating LaTeX for academic writing.

Online Tutorials and Guides

  • Overleaf’s LaTeX Tutorials: Overleaf offers a comprehensive series of tutorials for LaTeX beginners and advanced users alike, covering everything from basic document structure to complex mathematical typesetting.
  • RStudio’s R Markdown Documentation: The official R Markdown website by RStudio provides extensive documentation, tutorials, and galleries of examples to help users harness the full potential of R Markdown, including its LaTeX capabilities.

Community Forums and Support

  • Stack Exchange TeX — LaTeX Stack Exchange: A question and answer site for users of TeX, LaTeX, ConTeXt, and related typesetting systems. It’s an excellent resource for getting help with specific LaTeX questions or issues.
  • RStudio Community: The RStudio Community forum is a great place to ask questions and share insights about using R Markdown and LaTeX.

Packages and Tools

  • tinytex: An R package that provides a lightweight, portable, and easy-to-maintain LaTeX distribution. It’s specifically designed to simplify the management of LaTeX distributions in R Markdown workflows.
  • LaTeX Workshop for Visual Studio Code: For users who prefer Visual Studio Code as their editor, this extension enhances the LaTeX experience with features like build automation, comprehensive linting, and preview.

While we’ve covered substantial ground in this guide, the journey to mastering LaTeX in R Markdown is ongoing. The resources listed in this chapter offer pathways to further exploration and mastery. Whether you’re looking to refine your document designs, tackle complex typesetting challenges, or simply stay updated on new packages and features, the LaTeX and R Markdown communities offer a wealth of knowledge and support.

Remember, the key to proficiency in LaTeX and R Markdown is practice and engagement with the community. Don’t hesitate to experiment with your documents, ask questions, and share your knowledge with others. With these resources at your disposal, you’re well-equipped to take your document creation skills to new heights.


Crafting Elegant Scientific Documents in RStudio: A LaTeX and R Markdown Tutorial was originally published in Numbers around us on Medium, where people are continuing the conversation by highlighting and responding to this story.

To leave a comment for the author, please follow the link and comment on their blog: Numbers around us – Medium.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you’re looking to post or find an R/data-science job.


Want to share your content on R-bloggers? click here if you have a blog, or here if you don’t.

Continue reading: Crafting Elegant Scientific Documents in RStudio: A LaTeX and R Markdown Tutorial

Read the original article

RUGS: The Impact of R on Academic Excellence in Manchester

RUGS: The Impact of R on Academic Excellence in Manchester

[This article was first published on R Consortium, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)


Want to share your content on R-bloggers? click here if you have a blog, or here if you don’t.

The R Consortium recently spoke with the organizing team of the R User Group at the University of Manchester (R.U.M.). R.U.M. aims to bring together R users of all levels to share R best practices, expertise and knowledge. The group is open to all staff and postgraduate researchers at the University of Manchester, UK.

During the discussion, the team shared details about their recent events and their plans for this year. They also discussed the latest trends in the R programming language and how they are utilizing it in their work.

Martín Herrerías Azcué
Research Software Engineer
University of Manchester
Lana Bojanić
Researcher PhD Candidate
University of Manchester
Rowan Green
PhD Student in Evolutionary Microbiology 
The University of Manchester

Please share about your background and involvement with the RUGS group.

Martin: My name is Martin, and I joined the University of Manchester a year ago. They assigned me to manage the R user group, which was previously under Camila’s leadership. Although I am officially in charge, this is a collaborative effort between all of us who are present in this meeting, along with some others who couldn’t join. I work in Research IT and mainly use R for projects assigned to me by other people.

Anthony: My name is Anthony and I work at Research IT with Martin at the University of Manchester. I first came into contact with R when I was a student. Later, I became a helper at many of the university’s R training courses based on the Carpentries training courses. Camila, who was Martin’s predecessor, was also a trainer at R and she formed the R Users Manchester group. I volunteered to help her with the group a year ago, and it just turned a year old. After that, I continued to be a part of the group.

Lana: Hi there, my name is Lana. I am a PhD student and research assistant at the Division of Psychology and Mental Health at the University of Manchester. I have been using R for the past six years, ever since my Master’s degree. I have been a part of the group since its inception and have been running R introduction sessions for beginners within my division for a couple of years now. When I learned the group was being formed, I contacted Camila a year ago. This makes us founding members of the group. 

Rowan: Hello, my name is Rowan Green. I am currently a PhD student in the Department of Earth and Environmental Sciences. For my research work, I use R extensively for simulation modeling bacteria, analyzing lab data, and creating visualizations. The best thing about using R is that it produces much prettier visualizations than other options available to us as biologists. We have a lot of master’s and undergraduate students coming through the lab. I often give them pre-written scripts they can tweak to create their plots. It’s exciting to see them working hard to produce their plots.

Camilla mentioned starting a group to share knowledge about R on a university-wide level. I found this a great opportunity to participate and learn from others’ presentations during the meetings. It has been an enriching experience so far.

Can you share what the R community is like in Manchester? 

Anthony: In industries such as banking and finance, R is frequently used to create graphs to showcase econometric data in an easy-to-understand manner. The graphical capabilities of this programming language make it a popular choice in these fields. The university we’re in has access to the Financial Times, which is known for producing visually stunning graphs. Interestingly, they also use an R package called FT plot tools, which is a specialized package solely for their use. So, it’s safe to say that R has a significant presence in the banking and finance sectors. 

Are your meetups virtual or in-person? What topics have you covered recently? What are your plans for the group in the future?

Martin: Our events are a mix of in-person and online meetings. There have been talks about developing packages, data visualization, automating reports, and working with tables. We usually cover topics we are confident about or know people from the university are working on. However, we are also trying to get external speakers to come and talk. It’s challenging, but we are doing our best to make it happen. We are currently accepting proposals from potential speakers.

Our book club has mostly or completely taken place online.

Lana:  Bookclub was mostly online. During the summer book club, we were reading R for Data Science. We covered a chapter or two chapters each time. We had the book’s second edition, and people from all over the university joined the club.

We were discussing the possibility of changing the format of Tidy Tuesdays. We received feedback that people don’t have enough time to come up with something extra creative every month. Additionally, there has been a need for more practice. Therefore, we plan to redesign Tidy Tuesdays to be more practice-oriented than creativity-oriented. We will be implementing these changes this year.

Anthony: We’ve recently had several discussions on useful packages, particularly in R. Some packages that were developed and published were custom-made. We also had presentations on the cosinor and cosinor2 packages, which are used for fitting curves, and an R update package for validating clinical prediction models.

There are two other R groups in Manchester. Our aim for this year is to establish communication with them and collaborate in a coordinated manner. (Editor’s Note: We recently talked with the Manchester R User Group.) Currently, our group solely focuses on the internal R community at the University of Manchester.

Any techniques you recommend using for planning for or during the event?

Rowan: I’m not sure if everyone would agree with me, but I think we did well in the format of our meetings. We started with brief, brief talks – within an hour – followed by questions and discussions, which worked well. 

However, the harder part has been promoting and informing people about the meetings. Sometimes, word of mouth has been more effective than emails and posters. I noticed that they were interested in attending when I encouraged my lab group, who all use R. But without any scheduled reminders and someone to encourage them, it may be difficult to get people to come.

Lana: It’s important to identify everyone’s strengths or specialties within the organizing group, as they will probably be useful in the first few events. After that, you can expand your network within the community, which is easy to do since people are easily reachable. This will allow you to find interesting topic ideas and strengths to draw from.

What trends do you currently see in R language?

Martin: I’ve noticed a growing interest in Shiny lately, as I manage a pilot server for the university and have seen an increase in users over time. There have also been several inquiries about using R within our high-performance computer cluster, which may be something we can offer to the university. This interest is not surprising, given the current hype around machine learning.

A trending area that applies to multiple platforms, not just R, is towards reproducible research and compatibility between different programming languages. This means that R can be integrated with Python and other languages to create a documented and integrated pipeline. I’ve been experimenting with SnakeMake, which works well with R, but it would be great to see more integration from the R side, perhaps through the common workflow language or another similar tool.

Please share about a project you are currently working on or have worked on in the past using the R language. Goal/reason, result, anything interesting, especially related to the industry you work in?

Rowen: Recently, I wrote a preprint of a paper where we simulated the growth and mutation of bacteria using differential equations and R programming language. To perform the simulation, we utilized high-performance computing, which enabled us to simulate various ways the bacteria could grow by adjusting the rates of reactions occurring within the cells. This simulation required high-performance computing to be feasible for running multiple simulations.

After running simulations, we came up with some ideas to test in the lab. Our focus was on measuring mutation rates, and we used statistical analysis to estimate them through R. We have been striving to ensure reproducibility, and as a result, we have annotated all the data tables and R scripts with the paper.

It has been an interesting journey for me. I had to tidy up my messy scripts and think about how someone else would perceive them. I had to ensure they made sense. However, the project was fascinating as I generated hypotheses using R, tested them, and analyzed and visualized them with the same tool. R is a complete tool that can handle all aspects of the process, making it a brilliant choice.

How do I Join?

R Consortium’s R User Group and Small Conference Support Program (RUGS) provides grants to help R groups organize, share information, and support each other worldwide. We have given grants over the past four years, encompassing over 68,000 members in 33 countries. We would like to include you! Cash grants and meetup.com accounts are awarded based on the intended use of the funds and the amount of money available to distribute.

The post The Impact of R on Academic Excellence in Manchester, UK appeared first on R Consortium.

To leave a comment for the author, please follow the link and comment on their blog: R Consortium.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you’re looking to post or find an R/data-science job.


Want to share your content on R-bloggers? click here if you have a blog, or here if you don’t.

Continue reading: The Impact of R on Academic Excellence in Manchester, UK

Long-term Implications and Future Developments of R Programming Language in Academia

The R User Group at the University of Manchester (R.U.M.) is setting a significant trend in encouraging the collaboration of R Users to share best practices, expertise, and knowledge. They are not only building a community within the University of Manchester but are also making considerable strides in expanding and facilitating such groups globally.

Popularity of R Programming in Various Sectors

As discussed by Anthony, a member of R.U.M, R programming has increasingly become popular in fields such as banking and finance. This is largely due to its graphical capabilities which make it easier for users to present econometric data efficiently and comprehensibly. Even globally known platforms such as the Financial Times are using R packages like FT plot tools to produce visually stunning graphs. It is evident that R has a significant presence in the market due to its versatile usage.

Future Developments with R in Academia

The R.U.M group has diverse plans for the future. They aim to expand the group by having a mix of in-person and online meetings. R.U.M is also planning to invite external speakers to share their expertise. Additionally, they are planning to get more practice-oriented by redesigning their Tidy Tuesday event, moving away from a more creativity-oriented approach.

Integration and Collaboration with Other R User Groups

The group is planning to establish communication and collaboration with other R User groups in Manchester. The aim is to link all the R users and create a community that benefits everyone involved.

Trending Areas and Long-term Implications

Martin, the Research Software Engineer at the University of Manchester noted the trending area in R is towards reproducible research and compatibility with other languages such as Python. This multi-platform compatibility means R users have more resources and flexibility when working on their projects or research.

Applying R in Real-world Scenarios

The practical applicability and the beneficial impact of R were illustrated quite effectively by Rowan Green, a PhD student who used R for simulating the growth and mutation of bacteria. Using R from hypothesis generation to data visualization underlined the versatility and comprehensiveness of this programming language.

Actionable Advice

If you’re an R user, whether a novice, intermediate or expert, being a part of such groups would not only help you stay updated on the latest trends and developments in R, but it would also provide the opportunity to learn from other users’ experiences. The group encourages potential speakers to reach out and share their expertise, so if you’re confident in a particular topic or have some unique insights to share, consider getting involved.

Finally, take leverage of the growing interest in tools like Shiny and the increasing integration of R with other programming languages. This presents a considerable opportunity to broaden your skillset and makes you more adaptable in various research or project environments.

Read the original article

Comparing Data Visualization Tools: R Shiny, PowerBI, Spotfire

Comparing Data Visualization Tools: R Shiny, PowerBI, Spotfire

[This article was first published on Tag: r – Appsilon | Enterprise R Shiny Dashboards, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)


Want to share your content on R-bloggers? click here if you have a blog, or here if you don’t.

Data visualization and analytics tools are crucial for businesses and researchers alike. Power BI, Spotfire, and R Shiny have emerged as significant players in the market.

This article aims to compare these data visualization tools for businesses across various parameters, helping you make informed decisions based on your specific dashboard needs.

See how R Shiny stacks up against Tableau as an Excel alternative. Decide which tool fits your needs in our in-depth comparison: Tableau vs. R Shiny: Which Excel Alternative Is Right For You?

TL;DR:

  • This article compares R Shiny, Power BI and Spotfire, focusing on aspects like ease of use, customization, functionality, cost and performance.
  • R Shiny: High flexibility and customization, ideal for advanced analytics, requires R programming skills.
  • PowerBI: User-friendly for non-technical users, has good scalability and performance, and integrates well with other Microsoft products.
  • Spotfire: Offers robust analytics capabilities, excellent for handling complex data sets, and higher learning curve.
  • Choosing the right tool is based on your project’s needs, user skills, and organizational infrastructure. ‍

Data Visualization Business Tools – The Overview

R Shiny

Shiny, an R package, is renowned for its ability to create interactive web applications directly from R. One of its core features is the seamless integration of powerful data visualization libraries such as ggplot2 and plotly, which enable users to construct sophisticated and dynamic visual representations of data.

The framework is designed to be accessible, allowing even those without web development experience to produce professional-looking applications. Shiny applications are inherently reactive; they automatically update outputs when inputs change, without requiring a page refresh. This reactivity is a cornerstone of Shiny’s interactive nature.

Shiny also adopts a modular approach to application development, enabling users to craft complex UIs using its core UI functions. These functions allow for the creation of engaging and visually appealing user interfaces without the necessity for direct HTML or CSS coding, simplifying the development process while offering extensive customization through custom server logic. You can find our open-source and fully customizable Shiny templates on our website.

Below is a list of key features that make Shiny a standout choice for data analysis:

  • Reactive programming model
  • Integration with R’s rich ecosystem of packages
  • Customizable user interfaces
  • Ability to host and share applications on the web
  • Extensive resources and community support
Shiny Gosling – interactive genomics data visualisation

Explore the strengths of Python Dash and R Shiny to make an informed decision for your future projects. Learn more in our detailed guide: Python Dash vs. R Shiny – Which To Choose in 2021 and Beyond.

Power BI

PowerBI stands out in business intelligence for its robust integration with other Microsoft products and its user-friendly interface. One of the key capabilities of PowerBI is its Advanced AI features, which allow users to leverage artificial intelligence within their dataflows, enhancing the analytical power at their disposal.

The service is designed to accommodate a range of business scenarios, from small-scale reporting to enterprise-level data analysis. PowerBI’s ability to perform asynchronous refresh operations ensures that data is up-to-date without impacting system performance, a critical aspect for businesses that rely on real-time data insights.

PowerBI’s integration capabilities are further highlighted by its seamless connectivity with various data sources, both on-premises and in the cloud. This flexibility is crucial for organizations that manage diverse data ecosystems. PowerBI allows users to embed R/Python code, so this might be a neat benefit for someone with programming experience.

CEO dashboard

Compare two leading Excel alternatives, Power BI and R Shiny. Make an informed choice for your data analytics needs. Dive into our comprehensive analysis: Power BI vs R Shiny: Two Popular Excel Alternatives Compared.

Spotfire

Spotfire stands out as an enterprise-level data visualization tool due to its analytical depth and flexibility. Users can delve into complex data analysis with a platform designed to handle vast datasets and sophisticated algorithms. Spotfire’s flexibility is evident in its ability to integrate with various data sources and its support for numerous data formats.

Its powerful in-memory processing enhances its analytical capabilities, which allows for real-time data exploration and discovery. This is particularly beneficial for organizations that require immediate insights from their data. The tool’s flexibility extends to its visualization options, which include a wide range of charts, graphs, and interactive dashboards.

Spotfire’s extensibility is another key advantage. Users can extend the platform’s functionality with custom applications and integrations, making it a versatile choice for businesses with specific analytical needs. Below is a list highlighting some of Spotfire’s flexible features:

  • Integration with R, Python, and MATLAB for advanced analytics
  • Customizable dashboards and interactive visualizations
  • Support for real-time and historical data analysis
  • Robust options for predictive and big data analytics
Hospital Management

R Shiny vs. Power BI vs.  Spotfire – Which Data Visualization Tool is Best for Your Business Needs?

The table you’re about to see showcases a comparative analysis between our three tools of choice:

R Shiny PowerBI Spotfire
Flexibility and Customization R Shiny excels in customization, allowing users to create highly tailored and interactive dashboards. 

This allows for extensive customization at potentially lower costs, especially if the in-house team has R programming skills.

PowerBI offers a balance between ease of use and customization. 

It provides a user-friendly interface for creating custom reports and dashboards. Customizations might incur additional costs if there is a need for advanced analytics features or third-party integrations.

Spotfire offers a good balance of flexibility and customization with its advanced data visualization capabilities. 

Customization costs can be high, particularly for complex data models and advanced analytics features.

Advanced Analytics and Visualization R Shiny, being based on R, is powerful for statistical analysis and advanced analytics. It allows for customized analytics solutions, making it versatile for specific needs. PowerBI has improved its analytics features over time and is a strong choice for businesses requiring analytics and visualization, especially when integrated with Microsoft tools. Spotfire offers advanced analytics and visualization features. It provides robust predictive modelling and data mining capabilities.
Usability for Non-technical Users R Shiny may have a steeper learning curve, but its versatility becomes apparent once mastered, catering to the needs of both technical and non-technical users. PowerBI is known for its ease of use, particularly for business users who may not have extensive technical backgrounds. Spotfire provides a user-friendly interface, making it accessible to both technical and non-technical users.
Scalability, Performance, and Development Speed R Shiny is highly scalable and can be optimized for performance. However, development may take longer due to the need for R programming expertise. PowerBI is scalable, especially in Microsoft-heavy environments, and offers good performance. Its development speed is decent, particularly for those familiar with the Microsoft ecosystem. Spotfire is scalable and performs well in various business sizes. Its development speed is relatively fast due to its intuitive interface.
Cost-Effectiveness R Shiny, being open-source, is often the most cost-effective option in the long run, especially for organizations with skilled R programmers. PowerBI offers competitive pricing and can be cost-effective, especially for organizations already invested in Microsoft technologies. Spotfire can be expensive, especially for small to mid-sized organizations, which might impact long-term cost-effectiveness.
Cost of Maintenance Maintenance of Shiny apps requires regular updates to the R environment and packages. Due to its open-source nature, it might need more hands-on maintenance, especially for custom-built applications. 

The cost of maintenance can vary depending on the complexity of the app and the need for specialized R programming expertise.

PowerBI, being a Microsoft product, typically has a more streamlined update and maintenance process. However, the cost of maintenance could be higher due to licensing fees and the need for ongoing subscriptions for premium features. Spotfire offers robust support and regular updates as part of its enterprise-grade solution. The cost of maintenance is generally higher due to its positioning as a premium product, but it offers strong support and integration capabilities.

Discover the key differences between SAS and R programming and decide which is right for you. Check out our detailed comparison: SAS vs R Programming: Which to Choose and How to Switch.

R Shiny vs. PowerBI vs. Spotfire – Addressing the Limitations

No data dashboarding tool for business is perfect, and you must be aware of the limitations before pulling the trigger. Here are a couple of things you should be aware of:

Shiny:

  • While Shiny can handle large datasets, optimizing performance for these scenarios requires advanced R coding and server management skills.
  • While it’s possible to create aesthetically pleasing apps, achieving a high level of design polish may demand additional time and expertise in UI/UX design.
  • Shiny is a powerful tool, but it requires users to be proficient in R.

PowerBI:

  • PowerBI’s data modeling capabilities, while robust, might be limited in handling highly complex statistical analyses, which are better suited to specialized analytics tools.
  • Customization in PowerBI, though user-friendly, can be limited for specific or advanced requirements, potentially requiring additional tools or workarounds.
  • Dependency on Microsoft’s ecosystem could pose challenges in integration with certain non-Microsoft technologies or platforms.

Spotfire:

  • Despite its powerful analytics capabilities, Spotfire might not be the best choice for projects where simple data visualization is required, due to its complexity and cost.
  • The learning curve for effectively utilizing Spotfire’s advanced features can be steep, particularly for users without a background in data analytics.
  • Spotfire’s licensing and infrastructure costs can be significant, making it less accessible for smaller organizations or projects with limited budgets.

Finding the Best Data Visualization Tool for Your Organization

When assessing the best value for your organization, it’s crucial to look beyond the sticker price of data visualization tools. Consider the total cost of ownership (TCO), which includes not only the initial licensing fees but also the long-term costs associated with training, maintenance, and upgrades. A tool that seems inexpensive at first might require significant investment in these areas over time.

PowerBI and Spotfire offer different licensing models that cater to various organizational sizes and needs, and Shiny on the other hand is free to use. To determine which tool offers the best value, organizations should compare the features and support against their specific requirements. Here’s a simplified comparison:

  • Shiny: Free and open-source; ideal for R users and custom development. Offers subscription tiers for deploying applications to shinyapps.io.
  • PowerBI: Subscription tiers; integrates well with other Microsoft products.
  • Spotfire: Enterprise-level pricing; offers deep analytical capabilities.

‍Uncover which data dashboard tool is right for your projects: Choosing the Right Data Dashboard Tool: The Unique Strengths of Streamlit and Shiny.

Summing up R Shiny vs. PowerBI vs. Spotfire

In summary, Shiny, PowerBI, and Spotfire each offer unique strengths that cater to different business intelligence needs.

Shiny excels with its customizability and integration with R, making it ideal for statisticians and data scientists. PowerBI stands out for its user-friendly interface and deep integration with other Microsoft products, which is great for organizations entrenched in the Microsoft ecosystem. Spotfire, with its powerful analytical capabilities and real-time data exploration, is well-suited for enterprises requiring advanced analytics.

Ultimately, the choice between these tools should be guided by the specific requirements of the project, the technical proficiency of the users, and the existing infrastructure of the organization. By carefully considering these factors, businesses can leverage the right tool to transform their data into actionable insights and drive informed decision-making.

Did you enjoy this blog post and want to delve deeper into using Shiny for enterprise dashboards? Secure your spot at ShinyConf 2024 now and start exploring!

The post appeared first on appsilon.com/blog/.

To leave a comment for the author, please follow the link and comment on their blog: Tag: r – Appsilon | Enterprise R Shiny Dashboards.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you’re looking to post or find an R/data-science job.


Want to share your content on R-bloggers? click here if you have a blog, or here if you don’t.

Continue reading: Choosing the Right Data Visualization Tool: R Shiny, PowerBI or Spotfire

Business Intelligence: R Shiny vs Power BI vs Spotfire

Data visualization and analytic tools play a vital role in businesses and research. Among many tools available, Power BI, Spotfire, and R Shiny have emerged as significant contenders. This comprehensive comparison touches upon the core aspects of these tools, helping businesses make informed decisions based on their specific needs.

Contacting the Comparison

The comparison aims to help your organization choose the best tool for your specific dashboard needs. The focus is on ease-of-use, cost, performance, functionality, and customization of R Shiny, Power BI, and Spotfire.

R Shiny

Incorporating R Shiny into business allows for flexibility and customization. It is ideal for advanced analytics but requires R programming skills, making it a versatile option for data visualization needs, irrespective of the technical proficiency of the user.

Power BI

Power BI offers a user-friendly interface for non-technical users. Couples with good scalability, it delivers sound performance and integrates seamlessly with other Microsoft products, offering a solid platform for data visualization and analysis in a Microsoft-heavy environment.

Spotfire

Spotfire provides robust analytic capabilities and exhibits excellent prowess in handling complex data sets. However, it has a steep learning curve, making it more suitable for those with advanced data analysis requirements.

Long-Term Implications and Future Developments

In the long run, an organization’s choice of these data visualization tools will impact project execution, analytics, and ultimately decision-making. These tools are likely to experience continued growth and improvements, delivering a more robust collection of capabilities to handle increasingly complex data-sets.

Looking Ahead

Consider user skills, organizational infrastructure, and project requirements while selecting a data visualization tool. Future developments in these tools are likely to include new features for handling larger datasets, improved performance and scalability, and more seamless integration with other core business technologies.

Actionable Advice

Given the long-term implications of tool selection, businesses should invest time to evaluate several factors: the project’s specific dashboard needs; skill levels of users; and the organization’s existing infrastructure. While Power BI is user-friendly and has good scalability, Spotfire could be ideal for handling complex data sets. On the other hand, R Shiny is excellent for advanced analytics needs. Choose the tool that caters to most of your organization’s needs.

If the in-house team has R programming skills, R Shiny might be more cost-effective in the long run. Spotfire, despite being expensive, could be worth the cost for complex data models. On the other hand, PowerBI might provide an optimal balance between cost and features, especially for organizations already using other Microsoft products.

Lastly, look beyond the sticker price – consider total cost of ownership to find the best value for your organization.

In summary, all of these tools offer powerful capabilities for business intelligence and data visualization. Your choice should primarily be guided by your business’s specific needs, rather than trying to adapt your business to fit the tool. By carefully considering your specific requirements and understanding the strengths and weaknesses of each platform, you can leverage the best tool for your data visualization needs.

Read the original article

Seven Important Generative AI and Machine Learning Concepts Visually Explained in one-minute Data Animations

Understanding Generative AI and Machine Learning Through Visual Animations

The initial text does not provide an overview of the seven important generative AI and machine learning concepts visually explained in one-minute data animations. However, visual animation as an educational tool for complex subjects like AI and machine learning could significantly improve comprehension and retention of such concepts. This article will deliberate the potential long-term implications of such an approach, consider predicted future developments, and provide advice on how these insights can be leveraged.

Long-term Implications

Presenting multi-layered AI and machine learning concepts in a visually engaging manner holds the promise of wider accessibility and comprehension, bridging gaps between subject matter experts and novices or laypeople. The ability to understand these technologies means that more people can contribute to their evolution, strengthening the potential for innovation and progress.

Potential Future Developments

As visual learning becomes more prevalent, developments could extend beyond the purely academic sphere. We could see more usage of animated explanations within technology, finance, and other industries reliant on data-heavy technical concepts. Additionally, with the permeation of AR and VR technologies, immersive data animations could be the next frontier.

Actionable Advice

Given the potential benefits and developments detailed above, here are a few recommendations regarding to ensure the effective use of data visualizations:

  1. Embrace Visual Learning: Whether you’re a researcher, student, teacher, or industry professional, explore and adopt visual learning techniques for complex subjects like AI and machine learning.
  2. Invest in development: Businesses and educational institutions should consider investing in the development of visual learning tools that can simplify complicated concepts.
  3. Collaborate cross-industry: Collaboration between the technology, education and animation sectors is essential to create high-quality, accurate and effective visual learning tools.
  4. Stay tuned to new trends: Keep abreast with evolving trends in visual learning technology, such as AR and VR, to remain at the forefront of innovation.

In summary, the use of visual animations to explain complex AI and machine learning concepts holds immense promise. Harnessing this tool could lead to greater comprehension, innovation, and progress in these critical fields.

Read the original article