Skip to main content

One post tagged with "Kubeflow"

View All Tags

· 6 min read
Raluca Constanda

Intro​

As a full-stack developer with limited exposure to the AI development world, I attended the AI Dev Summit in Paris with a slight sense of being an outsider. My goal was simply to learn something new and stay updated with industry trends.

However, after the first two or three presentations, I realized that the conference catered to people like me—curious developers eager to explore new technologies and tech professionals facing new challenges as AI becomes increasingly integral to modern businesses.

My previous interactions with AI were limited to ChatGPT discussions, a few Bard chats, and using GitHub Copilot. So, consider this article a beginner-friendly perspective on what transpired at the AI Dev Summit in Paris last week.

The summit focused on open-source solutions for Generative AI and Large Language Models. This emphasis gave me a sense of technology democratization, especially since, until now, I had only heard of AI developments coming from the largest tech companies.

Mistral AI​

The first day started with a presentation from Mistral AI, a French company important in the LLMs market. They offer a chatbot and a platform for building apps. Mistral AI combines different AI models and business approaches to meet both commercial and community needs. It's important to note that their "open source" models can only be used for testing, research, personal, and non-production purposes. They are free for learning, but you have to pay if you want to use them for business. You can find more information at Mistral AI.

InstructLab​

InstructLab is a "truly" open-source project that facilitates the creation of a generic LLM through community contributions. It introduces a new way of developing LLMs by defining "recipes" for contributions. Open source developpers can send data as skills or knowledge, and the AI pipeline combines them to update the model. Submissions are made using simple YAML files. Each pull request must pass a "red flag" test and is then manually verified by a human to ensure the data is correct and relevant. For more details, visit InstructLab.

Vector databases, the AI DB​

Vector DBs appeared as a need for AI to have specific database functionalities like storing and filtering large scale similar data.

In simple terms, a vector is a list of numbers. Data like text and images are converted into vectors and then indexed. Similar vectors represent similar meanings, so unlike traditional databases, data is organized by meaning. This allows for fast but approximate searches, which is ideal for AI applications.

For developers, using these databases is straightforward. Vector databases integrate with most LLM providers, and the translation of data into vectors happens in the backend. They also work well in distributed and replicated systems. For a comparison of different vector databases, visit here.

The Openness of LLMs​

An interesting presentation discussed how open AI models are. This has evolved as companies try to balance business needs with community interests.

The "openness" of an AI model depends on a few factors:

  • Whether the data or assets are public
  • Any limitations on usage
  • Whether it's only described in research papers

Big players like OpenAI, Google, and Meta started with open research and full access. Later, they shifted to a more pragmatic approach, using licenses, limitations, or closing off data for newer versions.

Companies want to protect their work, especially since AI tools can be used for both good and bad purposes, but also because it involves huge financial investments. Most "open source" models have usage restrictions, such as:

  • Not for use in production
  • Restrictions on generating substantial income
  • Prohibitions on deployment in certain industries (e.g., military)

For example, early versions of OpenAI required that their output not be used to train similar models.

So, if you want to use an LLM, it's important to read the terms and conditions before building anything with it.

Foundation Models Challenges​

The first day ended with a discussion about the challenges Foundation Models (like LLMs) have brought to the software industry. Each of these challenges, from legal and ethical considerations to organizational and practical development, is extensive enough that they couldn't be summarized in a 30-minute presentation. They impact all business levels, which can be quite overwhelming. If you're interested, you can access the full article here.

Professor Ahmed E. Hassan highlighting the accuracy of prompts with different text formatting.

Professor Ahmed E. Hassan highlighting the accuracy of prompts with different text formatting.

Hugging Face: The GitHub of AI​

I started the second day with a presentation on Hugging Face Accelerate. Hugging Face is a collaborative platform for AI models and datasets.

The presentation focused on Accelerate, a framework for launching, training, and using PyTorch models. It requires minimal code and is easy to integrate with your existing models. The best part is that it allows you to run large models locally by efficiently using your resources. For more information, visit Hugging Face.

WasmEdge​

WasmEdge is a competitor to Docker and Kubernetes, offering a faster and lighter solution for containerization. This can be a big advantage for LLMs. WasmEdge is a runtime that lets you run LLMs locally without needing a powerful computer. For more information, visit WasmEdge on GitHub.

Community & Infrastructure​

The Cloud Native Compute Foundation has expanded with a new branch called Cloud Native Artificial Intelligence (CNAI). CNAI focuses on building and deploying AI applications and workloads using Cloud Native principles. It encompasses a wide array of technologies detailed here.

Two dedicated CNCF AI projects were highlighted:

  • Kubeflow: Facilitates machine learning (ML) workflow deployments on Kubernetes.
  • K8sGPT: A tool that scans Kubernetes clusters, identifies issues, and provides explanations in simple language using AI. This feature is tailored to juniors to enhance their understanding and usage of Kubernetes.

Additionally, the Linux Foundation hosts the Open Platform for Generative AI (OPEA), offering open-source solutions. Explore more at OPEA.

Conclusion​

After two days of presentations and chatbot demos, I left the AI Dev Summit with a sense of optimism. Using and developing Generative AI is no longer just for data scientists—it's becoming essential for any modern business and web platform. Software engineers now need to integrate and work with AI as part of their daily tasks. This shift has been evident in my own work with clients who are using AI to improve their businesses. Personally, I'm actively seeking AI solutions to streamline my own processes.

Another takeaway is the vibrant community around open-source AI projects. Both the community and companies are actively fostering learning, experimentation, and contributions.

Instead of a traditional conclusion, here are some GitHub links in case you're interested in running a Large Language Model (LLM) locally:

All presentations from the AI Dev Summit are accessible on Sched. Recordings of the sessions will also be available on the Linux Foundation's YouTube channel.