Getting Started with Local Language Models: A Tutorial

Getting Started with Local Language Models: A Tutorial

[This article was first published on Posts | Joshua Cook, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)

Want to share your content on R-bloggers? click here if you have a blog, or here if you don’t.

With so much hype around LLMs (e.g. Chat-GPT), I’ve been playing around with various models in the hope that when I come up with a use case, I will have the skill-set to actually build the tool.
For privacy and usability reasons, I’m particularly interested in running these models locally, especially since I have a fancy MacBook Pro with Apple Silicon that can execute inference on these giant models relatively quickly (usually just a couple of seconds).
With yesterday’s release of a new version of Code Llama, I figured it could be helpful to put together a short post on how to get started playing with these models so others can join in on the fun.

The following tutorial will show you how to:

  1. get set up with Ollama,
  2. create a Python virtual environment,
  3. and provide and explain a simple Python script for interacting with the model using LangChain.

Setting up Ollama

Ollama is the model provider.
Another popular option is HuggingFace, but I have found using Ollama to be very easy and fast.

There are multiple installation options.
The first is to just download the application from the Ollama website,, but this comes with an app icon and status bar icon that I really don’t need cluttering up my workspace.
Instead, I opted to install it with homebrew, a popular package manager for Mac:

brew install ollama

With Ollama installed, you just need to start the server to interact with it.

ollama serve

The Ollama server will run in this terminal, so you’ll need to open another to continue with the tutorial.
You’ll need to start up the server anytime you want to interact with Ollama (e.g. downloading a new model, running inference).

We can now interact with Ollama, including downloading models with the pull command.
The available models are listed here.
Some models have different versions that are larger or for specific use cases.
Here, we’ll download the Python-fine tuned version of Code Llama.
Note that there are also larger versions of this model that may improve it’s quality.

ollama pull codellama:python

That’s it!
We now have Ollama running and ready to execute inference on the latest Python Code Llama model.

Setting up the Python virtual environment

This is a routine process, not specific to LLMs, but I figured I’d include it here for those unfamiliar.
Below, I create a Python virtual environment, activate it, and then install the necessary LangChain libraries from PyPI.

python -m venv .env
source .env/bin/activate
pip install --upgrade pip
pip install langchain langchain-community

The above commands use the default version of Python installed on your system.
To exercise more control over the versions of Python, I use ‘pyenv’, though this is a bit more complicated and I won’t cover using it here.
It is worth mentioning though for those with a bit more experince.

Interacting with Code Llama using LangChain

“LangChain is a framework for developing applications powered by language models.”
It is a powerful tool for interacting with LLMs – scaling from very simple to highly complex use cases and easily swapping out LLM backends.
I’m still learning how to use it’s more advanced features, but LangChain is very easy to get started with.
The documentation has plenty of examples and is a great place to start with for learning more about the tool.

Here, I’ll provide the code for a simple Python script using LangChain to interact with the Python Code Llama model downloaded above.
I hope this offers a starting point for those wishing to explore playing with these models, but are overwhelmed by the myriad options available.

Note, that you need to have the Ollama server running in the background by executing ollama serve in another terminal (or already running from the previous step).

Below is the code for those who want to take it and run.
Following it, I have more information about what it is actually doing.

"""Demonstration of using the Python Code Llama LLM."""

from langchain.prompts import PromptTemplate
from langchain_community.llms import Ollama
from langchain_core.output_parsers import StrOutputParser

def main() -> None:
 prompt = PromptTemplate.from_template(
 "You are a Python programmer who writes simple and concise code. Complete the"
 " following code using type hints in function definitions:"
 "nn# {input}"
 llm = Ollama(model="codellama:python")
 output_parser = StrOutputParser()

 chain = prompt | llm | output_parser

 response = chain.invoke(
 {"input": "Request a wikipedia article and pull out the tables."}

if __name__ == "__main__":

If the above code is copied to a file, this script can be executed with the following:


There are three sections to this script:

  1. the import statements that load the relevant LangChain libraries,
  2. the main() function that executes the demo (described in detail below),
  3. and the if statement that executes the main() function if this file is run as a script.

The main() function holds the actual code for interacting with the LLM.
It starts by creating prompt, a LangChain Prompt that will take the input from the user and pass it to the model, first wrapping it with some instructions for the LLM.
The LLM object is then created, specifying the model with the same name we used to download it earlier in the tutorial.
The last component is just a simple output parser that converts the model’s output to a string for easy printing.

These three components are then combined into a chain using the pipe (|) operator that LangChain has overloaded to support it’s clever chaining syntax.
The chain’s invoke() method is then executed to pass a request to the LLM.
Note that a dictionary is passed with a key matching the keyword input in the prompt template.
The text passed as “input” will be inserted into the template and the result will then be sent to the LLM.


That’s pretty much it.
These few lines of simple code can get you up a running with an LLM running on your local machine!
I hope this has provided you with some guidance for getting started and was relatively easy to follow.
I would recommend getting the demo running and then perhaps playing with some variables such as:

  • experimenting with different prompts,
  • trying different types of tasks such as having the model inspect code for bugs for writing tests,
  • comparing the results from different models, such as larger Code Llama options, the general vs. Python-specific models, try swapping in a ChatGPT backend, or even figure out a use case for multi-modal models (e.g. llava).

For fun, here is the output from running the above code on the codellama:13b-python model and input “Download a wikipedia article on marmots and extract any tables it contains.”

# Save each table as a separate csv file in your working directory.

from bs4 import BeautifulSoup
import requests, re
def get_marmot_article(url):
 r = requests.get(url)
 html_content = r.text
 soup = BeautifulSoup(html_content,'lxml')
 return soup

def find_tables_in_soup(soup):
 tables = []
 for table in soup.findAll('table'):
 if 'class' in table.attrs:
 if table['class'] == 'wikitable':
 return tables

def extract_from_tables(tables, filename):
 dfs = []
 for i, t in enumerate(tables):
 headers =
 rows = [row.text.split('n')[0].strip()
 if len(row.text.split('n')) >=2 else ''
 for row in t.findAll('tr', recursive=False)][1:]

 data = list(zip(* + ))

 dfs.append(pd.DataFrame(data, columns=['Species','Fur color', 'Range']))
 dfs.to_csv('marmot_{}.csv'.format(i), index=False)
 return dfs
To leave a comment for the author, please follow the link and comment on their blog: Posts | Joshua Cook. offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you’re looking to post or find an R/data-science job.

Want to share your content on R-bloggers? click here if you have a blog, or here if you don’t.

Continue reading: Quickstart for playing with LLMs locally

The Future Potential of Local Language Models (LLMs): An Analysis

The rise of LLMs in software programming and machine learning has drawn significant attention recently. Such models, which include Chat GPT, for example, have multi-purpose applications and a rising relevance in diverse fields. In this follow-up analysis, we delve into the long-term implications and future developments around LLMs such as Code Llama and Ollama.

The Evolution of Language-Learning Models

The development of language-learning models has been significant over the last few years. With the possibility to run these models locally on hardware like Apple’s Silicon, their potential has only increased.

Increased focus on privacy and usability

As more organizations and individual developers recognize the importance of privacy and usability, LLM solutions that can be run locally are becoming increasingly popular. The localization creates a strong confidentiality barrier as compared to cloud-based tools and enhances model response times.

Potential for increased adoption

Given their simple setup process and the availability of resources like Code Llama and Ollama for beginners to learn from, LLMs have greater potential for wider adoption among new machine learning enthusiasts. The relative speed advantage presented by local machine execution enhancements like Apple Silicon could also be a major selling point down the line.

Future Developments to Watch For

  1. Advancements in local model execution: As hardware manufacturers increase focus on developing chips with large scale AI processing abilities, we could expect models that can be executed locally to become massively more efficient and time-saving than they are currently.
  2. Breadth of applications: As developers spend more time experimenting with LLMs and as their capabilities continue to grow, we could expect to see these models branching out into niche and specific use-cases beyond what’s currently imaginable.
  3. Improvements in LLM Frameworks: Tools like LangChain, which allow for user-friendly interaction with LLMs, can be expected to see advancements in terms of their functionality and ease of use, further propelling adoption among beginners.

Actionable Advice for Developers

  1. Experiment: Actively experimenting with these tools should be a priority for developers and students interested in machine learning. A hands-on experience with developing solutions using LLMs can prove highly advantageous as their relevance continues to grow.
  2. Stay updated: As with any other field within technology, staying up to date with the latest developments and advancements is key. Regularly check the updates and new releases from authorities like Code Llama and Ollama.
  3. Play with coding tasks: Coding tasks such as inspecting code for bugs or writing tests can be done using LLMs. This can help beginners improve their skill sets and grasp the practical applications of LLMs.
  4. Explore different models: Do not limit your knowledge to just one model. Take time to compare results from different models and eventually even swap backends to get a full understanding of the capabilities of each.

LLMs offer a wide range of potential applications and present a burgeoning field of study for software developers and machine learning enthusiasts. By taking a hands-on approach and plunging into the world of LLMs now, developers can future-proof their skillsets and stay at the forefront of technological innovation.

Read the original article

“Unlocking the Future of Data Analytics: Free Courses to Kickstart Your Career”

“Unlocking the Future of Data Analytics: Free Courses to Kickstart Your Career”

Looking to make a career in data analytics? Take the first steps today with these free courses.

The Future of Data Analytics: Insights and Opportunities

Data analytics is gaining traction as a lucrative and rewarding career, with businesses of all sizes banking on data-driven insights to streamline their operations and boost profitability. To seize these opportunities, it’s crucial to upskill with the help of versatile and extensive courses. Even better, there are now numerous accessible, free courses to help beginners embark on their data analytics journey. But what does this rise in data analytics mean for individuals and industries in the long term?

Long-term Implications for Businesses and Industries

As data becomes integral to decision-making processes in companies, data analytics skills are likely to grow in demand. Employers will look for professionals well-versed in data collection, management, interpretation, and projection to enable informed decision-making.

The emphasis on data analytics is not confined to specific industries – it holds relevance across sectors. Healthcare providers look to data to optimize patient care. Retailers analyze consumer behavior for targeted marketing campaigns. Governments interpret demographic metrics for policy formulation. Hence, the demand for data analytics expertise spans far beyond the realms of IT and tech companies.

Future Developments for Data Analytics

In the future, advancements in technologies like AI and machine learning will further transform the scope of data analytics. These technologies will enable more efficient data processing and trend prediction, enabling businesses to strategize even better based on insights drawn from intricate datasets.

Data privacy regulations will also play a crucial role in shaping how businesses handle and analyze data. This factor underscores the importance of obtaining a solid understanding of not just technical data handling skills, but also ethical considerations and legalities.

Actionable Advice: Navigating Your Way in Data Analytics

If you’re considering a career in data analytics, here are some steps to help you get started:

  1. Start with foundational skills: Take advantage of free courses available online. Gain a firm grounding in fundamental concepts like data collection, cleaning, exploration, and visualization.
  2. Choose your specialization: As noted above, data analytics skills are applicable across industries. Find a niche that interests you – be it healthcare, retail, government, or any other sector.
  3. Stay abreast of the latest trends: As data analytics is a rapidly evolving field, it’s crucial to stay updated. Follow thought leaders, join professional communities, and participate in webinars and workshops.
  4. Focus on both technical and ethical aspects: Mastering the handling of data is essential, but it’s also crucial to understand ethical considerations and legalities related to data privacy.

In conclusion, the rise of data analytics presents myriad opportunities for individuals and businesses. By investing time in gaining relevant skills and knowledge, you can position yourself to capitalize on the potential this field offers in the long-term.

Read the original article

Data governance is more important than ever in e-commerce, where massive amounts of data are generated and processed daily. Big Data presents opportunities and challenges for e-commerce businesses, requiring a strategic approach to data quality, security, and compliance. This article discusses e-commerce data governance best practices, including understanding data governance, data quality, data security, compliance… Read More »Mastering E-commerce data governance: Best practices, challenges, and future trends for quality, compliance, and growth

E-commerce Data Governance: A key determinant of future Business Success

Data governance has become an indispensable element of e-commerce, with vast quantities of data generated and processed each day. The growth of Big Data ushers several opportunities and challenges for e-commerce businesses, which necessitates a strategic approach towards data quality, security, and compliance. This article explores the concept of data governance in the perspective of e-commerce, particularly focusing on its best practices, challenges, potential future trends, and their implications.

Long-term implications

Data governance is not just about managing data; it involves structuring the data in such a way that it upholds data quality, security, and regulatory compliance in all aspects. These could have lasting implications for e-commerce businesses, shaping the way they function and strategically plan for future growth.

Data Quality: Strong data governance ensures high data quality which is crucial for making informed business decisions. Over the longer term, consistent efforts towards maintaining high-quality data can lead to greater operational efficiency, reduced errors and costs, which will substantially improve the overall performance of businesses.

Regulatory Compliance: Data governance protocols ensure regulatory compliance and reduce any legal risks associated with data non-compliance. Adherence to these rules can potentially save e-commerce businesses from hefty fines and tarnished reputation, thereby securing their long-term market position.

Possible Future Developments

Trends suggest that data governance will become more complex with advancing technology. Businesses must proactively anticipate future trends in order to evolve their data governance strategies accordingly.

  • Data Governance as a Service: The future may witness a rising trend in Data Governance as a Service (DGaaS), which involves outsourcing data governance tasks to external parties specializing in this field.
  • Integration with AI and Machine Learning: The integration of AI and Machine Learning with data governance can facilitate an automated, more efficient method for managing data.
  • Strengthening Data Privacy: With growing concerns over data privacy, future data governance trends are likely to stress upon safeguarding user data and implementing stronger data protection measures.

Actionable Advice

Understanding the potential implications and future developments of e-commerce data governance can help businesses prepare better. Here are a few pieces of advice:

  1. Target High-Quality Data: Businesses should aim to maintain high-quality data by implementing regular quality checks. This could improve operational efficiency in the long run.
  2. Ensure Compliance: Don’t neglect compliance rules as non-compliance could result in heavy fines and a damaged reputation. Keep up-to-date with the latest regulations.
  3. Prepare for Future Trends: Businesses should stay informed about future trends in data governance, and adapt their strategies accordingly. This proactive approach can effectively prepare businesses for what lies ahead.

In conclusion, e-commerce businesses should place high emphasis on robust data governance. By maintaining data quality, ensuring regulatory compliance, and keeping up with future trends, businesses can equip themselves for long-term growth in the e-commerce sector.

Read the original article

“Advancements in Moonwalk Simulations: Exploring VR, AI, and Inclusivity”

“Advancements in Moonwalk Simulations: Exploring VR, AI, and Inclusivity”

Advancements in Moonwalk Simulations: Exploring VR, AI, and Inclusivity

The text discusses the preparations for next moonwalk simulations that are currently underway, both on land and underwater. This article will analyze the key points of this text and discuss potential future trends related to these themes, as well as provide unique predictions and recommendations for the industry.

Moonwalk simulations have always been an important part of astronaut training and preparation for space exploration missions. These simulations allow astronauts to experience conditions similar to those they would encounter on the moon and help them practice and refine their skills. The text mentions that these simulations are not only taking place on land, but also underwater.

Underwater simulations have become increasingly popular in recent years due to their many advantages. Performing moonwalk simulations underwater allows astronauts to experience reduced gravity conditions similar to those on the moon. This helps them become familiar with the unique challenges and dynamics of moving and working in low-gravity environments. Additionally, water has a similar consistency to lunar soil, making it an excellent substitute for training purposes.

One potential future trend related to moonwalk simulations is the development of virtual reality (VR) technology for training purposes. VR technology has seen significant advancements in recent years and has the potential to revolutionize astronaut training. By creating immersive and realistic virtual environments, astronauts can practice moonwalk simulations without the need for physical resources or specialized facilities. This could significantly reduce costs and increase accessibility to training programs.

Another potential trend is the integration of artificial intelligence (AI) into moonwalk simulations. AI technology can be used to create realistic simulations that respond dynamically to the actions of astronauts. AI algorithms can generate complex scenarios and adapt in real-time based on the performance and decisions of the trainees. This would allow astronauts to experience a wide range of challenging situations in a controlled and safe environment.

In terms of recommendations for the industry, it is crucial to continue investing in research and development of new technologies for moonwalk simulations. Advancements in VR and AI have the potential to greatly enhance training programs and improve astronaut preparedness. Collaboration between space agencies, academia, and technology companies is crucial to ensure the development of effective and comprehensive training solutions.

Furthermore, it is important to prioritize accessibility and inclusivity in astronaut training. Moonwalk simulations should be designed to accommodate individuals with different physical abilities and ensure equal opportunities for all aspiring astronauts. This includes considering the needs of astronauts with disabilities and providing appropriate accommodations and support systems.

In conclusion, the future of moonwalk simulations looks promising with the integration of VR and AI technologies. These advancements have the potential to enhance training programs, reduce costs, and increase accessibility. By investing in research and development and prioritizing inclusivity, the industry can ensure that future astronauts are well-prepared for space exploration missions.

– NASA. (n.d.). Preparations for Next Moonwalk Simulations Underway (and Underwater). Retrieved from *URL of the original text*

MT-HCCAR: Multi-Task Deep Learning with Hierarchical…

MT-HCCAR: Multi-Task Deep Learning with Hierarchical…

In the realm of Earth science, effective cloud property retrieval, encompassing cloud masking, cloud phase classification, and cloud optical thickness (COT) prediction, remains pivotal….

Cloud property retrieval is a crucial aspect of Earth science, encompassing various elements such as cloud masking, cloud phase classification, and cloud optical thickness (COT) prediction. This article explores the importance of effective cloud property retrieval in understanding and analyzing Earth’s atmosphere. By accurately assessing these properties, scientists can gain valuable insights into climate change, weather patterns, and other atmospheric phenomena. With advancements in technology and data analysis techniques, researchers are striving to improve the accuracy and efficiency of cloud property retrieval methods.

In the realm of Earth science, effective cloud property retrieval, encompassing cloud masking, cloud phase classification, and cloud optical thickness (COT) prediction, remains pivotal. Understanding and accurately characterizing clouds is crucial for a variety of applications, including weather forecasting, climate modeling, and remote sensing. However, the complexity of cloud behavior and the inherent challenges in remote sensing make it a difficult task.

Unveiling the Mysteries of Cloud Properties

Clouds are dynamic and diverse, presenting a spectrum of shapes, sizes, and properties. They play a significant role in the Earth’s energy budget by reflecting sunlight back into space and trapping heat near the surface. Therefore, obtaining precise information about cloud properties is fundamental.

Cloud masking is the first step in cloud property retrieval, aiming to distinguish between cloudy and clear-sky regions. This task is challenging due to the presence of thin clouds, sub-pixel clouds, and cloud contamination caused by atmospheric aerosols. Traditional methods rely on spectral thresholds or statistical techniques to identify clouds. However, these approaches may result in false positive or false negative detections.

Cloud phase classification involves determining whether a cloud is composed of liquid water droplets or ice crystals. Accurate phase identification is crucial for understanding cloud processes and their effects on precipitation and radiation. Existing algorithms utilize infrared and microwave observations to differentiate between liquid and ice clouds. However, improvements are needed to handle mixed-phase clouds and accurately identify the boundaries of cloud phases.

COT prediction entails estimating the thickness or optical depth of clouds. This property determines how much sunlight is absorbed or scattered by a cloud layer. Accurate COT retrieval is vital for assessing the impact of clouds on climate and weather patterns. Most COT estimation techniques rely on radiative transfer models and observations from multiple spectral bands. However, uncertainties in radiative transfer calculations and measurement errors make it challenging to achieve robust predictions.

Championing Innovation for Improved Cloud Property Retrieval

To address the challenges in cloud property retrieval, innovative solutions and ideas are essential. Harnessing the power of advanced technologies and interdisciplinary collaborations can pave the way for significant advancements in this field. Here are some potential approaches:

  1. Machine Learning: Leveraging machine learning techniques can enhance cloud masking by training algorithms on large datasets with precise cloud identification. Deep learning algorithms can extract complex features from multi-spectral observations, improving cloud detection accuracy.
  2. Novel Remote Sensing Instruments: Developing new sensors that capture a wider range of spectral information can aid in better cloud phase classification. Incorporating advanced polarimetric measurements and active remote sensing techniques, such as lidar, can provide valuable insights into cloud microphysical properties.
  3. Fusion of Multiple Data Sources: Integrating information from various sensors, including visible, infrared, and microwave bands, can lead to more accurate COT predictions. Combining passive and active remote sensing observations with meteorological data can improve the understanding of cloud dynamics and their impact on Earth’s climate system.
  4. Collaboration and Data Sharing: Encouraging collaboration among researchers, institutions, and space agencies is vital for progress. Sharing data, methodologies, and validation exercises can foster innovation and enable the development of robust cloud property retrieval algorithms.

Cloud property retrieval plays a critical role in advancing our understanding of Earth’s climate system. By embracing innovation and collaborative efforts, we can unlock the mysteries of clouds and pave the way for more accurate weather predictions, improved climate models, and enhanced remote sensing capabilities.

The field of Earth science heavily relies on accurate cloud property retrieval for a variety of applications such as weather forecasting, climate modeling, and remote sensing. Cloud masking, cloud phase classification, and cloud optical thickness (COT) prediction are three key components of cloud property retrieval that play a crucial role in understanding and quantifying cloud characteristics.

Cloud masking is the process of distinguishing between cloudy and cloud-free areas in satellite imagery or other remote sensing data. Accurate cloud masking is essential to ensure that subsequent analysis focuses only on relevant cloud data. It involves the use of various algorithms and techniques to identify and remove non-cloud elements such as land, water bodies, or atmospheric artifacts.

Once clouds are identified, cloud phase classification comes into play. Clouds can exist in different phases, such as liquid droplets, ice crystals, or a mixture of both. Determining the phase of clouds is vital for understanding their impact on Earth’s energy balance and precipitation processes. Advanced algorithms utilizing multiple satellite observations and various spectral measurements are employed to classify cloud phase accurately.

Cloud optical thickness (COT) prediction is another critical aspect of cloud property retrieval. COT provides information about the amount of solar radiation that clouds can absorb or reflect. It serves as a key parameter for estimating the radiative properties of clouds and their impact on climate. Predicting COT involves analyzing the interaction between clouds and electromagnetic radiation across different wavelengths, allowing scientists to derive estimates of cloud thickness.

Moving forward, advancements in technology, such as improved satellite sensors and computational capabilities, will likely enhance the accuracy and efficiency of cloud property retrieval. Machine learning algorithms and artificial intelligence techniques hold great promise for automating and refining the process of cloud masking, phase classification, and COT prediction. These techniques can leverage vast amounts of data to train models that can rapidly and accurately analyze complex cloud patterns.

Furthermore, ongoing research aims to develop synergies between different Earth observation platforms, combining data from satellites, ground-based sensors, and airborne measurements. Integrating multiple data sources can provide a more comprehensive view of clouds and their properties, allowing for better understanding and prediction of weather patterns, climate change, and their impacts on ecosystems.

In conclusion, effective cloud property retrieval is essential for advancing our understanding of Earth’s climate system. Cloud masking, phase classification, and COT prediction are fundamental components that aid in quantifying cloud characteristics and their influence on various Earth science applications. Continued advancements in technology and data analysis techniques will likely lead to further improvements in cloud property retrieval, enabling more accurate weather forecasts, climate models, and remote sensing applications.
Read the original article