\n\n\n\n
1. Programming Language<\/strong><\/p>\n\n\n\n I tried to play with many languages and what I found was that most of the languages are overhyped. Yes, and that is when I met one of the communities talking about this new language made for cloud and AI applications. That was the first time I tried the Wing programming language and found it pretty impressive.<\/p>\n\n\n\n You might ask \u2014 why Wing? Wing offers a unified programming model that consolidates infrastructure and application code within a cohesive framework. This unique approach allows developers to streamline their workflow, eliminating the need for constant context switching and significantly enhancing productivity and creativity.<\/p>\n\n\n\n This is what exactly you need while building AI\/ML applications, focusing more on the core features rather than the underlying infrastructure. I encountered their Open AI Joker application, which generates and translates jokes into different languages. It was amazing to see how smooth the entire framework was. Just a note that Wing is still under active development.<\/p>\n\n\n\n You can build any AI\/ML application with minimal code. Let\u2019s see how the Joker application works. The application generates jokes using OpenAI and translates them to different languages. There is a comedian, an Open AI assistant that generates jokes and whenever it generates jokes, the joke gets put\/stored in a bucket. There are two translators, Spanish and Hebrew. These translators subscribe to a topic. Whenever a joke is generated, they receive it and translate it. Also, they put the translated joke into the bucket as well. Pretty simple.<\/p>\n\n\n\n Below is a technical overview image of this example of how to use OpenAI\u2019s API with Wing<\/a>.<\/p>\n\n\n\n If you haven\u2019t tried Winglang yet, you can easily try it with a simple command<\/p>\n\n\n\n Copy the code to your local computer using this git clone command<\/p>\n\n\n\n Go to the examples folder<\/p>\n\n\n\n Then run the following commands in the terminal:<\/p>\n\n\n\n Invoke the \u201cSTART HERE\u201d function, and see the results in the \u201cJoke Store\u201d.<\/p>\n\n\n\n For AI\/ML applications, you need a database to store the unstructured data. I joined SingleStore almost 8 months back and I was new to AI\/ML and vector databases. With all the hype around vector databases, I started learning about vector databases and found out how SingleStore is an amazing addition to the industry as a vector database. It is not just used to store the vector data but companies use SingleStore for real-time analytics also. Now, see the power of vector data storage supporting real-time analytics, that\u2019s a superpower.<\/p>\n\n\n\n Let me introduce SingleStore to you all; it is a cloud-based database management system (RDBMS) that is designed for data-intensive applications. It is known for its speed in data ingestion, transaction processing, and query processing. SingleStore started supporting vector storage long back in 2017<\/p>\n\n\n\n SingleStore\u2019s Notebook feature is based on the popular Jupyter Notebook, which is widely used in data science and machine learning communities. The SingleStore Notebook extends the capabilities of Jupyter Notebook to enable data professionals to easily work with SingleStore\u2019s distributed SQL database while providing great extensibility in language and data sources.<\/p>\n\n\n\n Try SingleStore and get $600 worth of free credits<\/a>.<\/p>\n\n\n\n I just love working with data and doing different experiments taking publicly available datasets like wine dataset, titanic dataset, etc to name a few. I was mesmerized to see the capabilities of NumPy and Pandas in data exploration and coming up with different solutions.<\/p>\n\n\n\n Pandas and NumPy are two of the most popular libraries in the Python ecosystem for data analysis and scientific computing<\/p>\n\n\n\n At the heart of any AI\/ML application is data. Tools like Pandas and NumPy are foundational for data manipulation and analysis in Python. Pandas provide high-level data structures and operations for manipulating numerical tables and time series, making it ideal for preprocessing and cleaning data before it\u2019s used for training models. NumPy adds support for large, multi-dimensional arrays and matrices, along with a large collection of mathematical functions to operate on these arrays, crucial for performance-heavy operations in data preprocessing and model training.<\/p>\n\n\n\n I have used TensorFlow, PyTorch and recently came across LangChain and LlamaIndex and was impressed by their ability to help AI\/ML engineers with all the toolkits required such as APIs, vector storage functionalities, logic, reasoning, etc to build robust applications. Let\u2019s go through them one by one to see their superpowers.<\/p>\n\n\n\n TensorFlow,<\/a> developed by Google, and PyTorch<\/a>, developed by Facebook, are two of the most popular frameworks for building and training complex machine learning models. TensorFlow is known for its flexibility and robust scalability, making it suitable for both research prototypes and production deployments. PyTorch is praised for its ease of use, simplicity, and dynamic computational graph that allows for more intuitive coding of complex AI models. Both frameworks support a wide range of AI models, from simple linear regression to complex deep neural networks.<\/p>\n\n\n\n Developed by Harrison Chase, and debuted in October 2022, LangChain<\/a> serves as an open-source platform designed for constructing sturdy applications powered by Large Language Models, such as chatbots like ChatGPT and various tailor-made applications.<\/p>\n\n\n\n Langchain seeks to equip data engineers with an all-encompassing toolkit for utilizing LLMs in diverse use cases, such as chatbots, automated question-answering, text summarization, and beyond.<\/p>\n\n\n\n LlamaIndex<\/a> is an advanced orchestration framework designed to amplify the capabilities of LLMs like GPT-4. While LLMs are inherently powerful, having been trained on vast public datasets, they often lack the means to interact with private or domain-specific data.<\/p>\n\n\n\n LlamaIndex bridges this gap, offering a structured way to ingest, organize, and harness various data sources \u2014 including APIs, databases, and PDFs. By indexing this data into formats optimized for LLMs, LlamaIndex facilitates natural language querying, enabling users to seamlessly converse with their private data without the need to retrain the models.<\/p>\n\n\n\n As a beginner, I was looking for something simple and flexible for developing deep learning models and that is when I found Keras<\/a>. Many AI\/ML professionals appreciate Keras for its simplicity and efficiency in prototyping and developing deep learning models, making it a preferred choice, especially for beginners and for projects requiring rapid development.<\/p>\n\n\n\n For developers looking for a high-level neural networks API, Keras, which is now integrated into TensorFlow, offers a simpler interface to build and train deep learning models. Keras abstracts away much of the complexity of building neural networks, making it accessible for beginners while still being powerful enough for research.<\/p>\n\n\n\n Collaboration and version control are crucial in AI\/ML development projects due to the iterative nature of model development and the need for reproducibility. GitHub<\/a> is the leading platform for source code management, allowing teams to collaborate on code, track issues, and manage project milestones. DVC<\/a> (Data Version Control) complements Git by handling large data files, data sets, and machine learning models that Git can\u2019t manage effectively, enabling version control for the data and model files used in AI projects.<\/p>\n\n\n\n I built some AI\/ML applications but how and where to deploy them? That is where my mind jumped to the two main tools in this category: Docker<\/a> and Kubernetes<\/a>. Like I said before, I come from DevOps parts of the world and I have already worked with these tools and I know how they work. While Docker containerizes your application, Kubernetes is used to deploy your applications for scale.<\/p>\n\n\n\n Deploying AI models into production requires tools that can package applications and manage them at scale. Docker simplifies the deployment of AI applications by containerizing them, ensuring that the application runs smoothly in any environment. Kubernetes, an orchestration system for Docker containers, allows for the automated deployment, scaling, and management of containerized applications, essential for AI applications that need to scale across multiple servers or cloud environments.<\/p>\n\n\n\n You cannot scale anything without cloud platforms like AWS, Google, and Azure<\/a>. While AWS is my favorite, I have also explored other options and mentioned all three major cloud providers here.<\/p>\n\n\n\n Cloud platforms like Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure offer a range of AI and machine learning services that abstract away much of the infrastructure required to train and deploy AI models. These platforms provide managed services for machine learning model training, deployment, and monitoring, along with a vast array of computational resources scalable to the needs of any AI project.<\/p>\n\n\n\n While I still prefer using the SingleStore Notebook feature, I know most of you are already using Jupyter Notebooks for your data explorations and analysis. I sometimes use Jupyter Notebooks and one more interesting tool is MLflow which will help you with end-to-end ML workflow.<\/p>\n\n\n\n For exploratory data analysis, model development, and documentation, Jupyter Notebooks are an indispensable tool. They allow developers to create and share documents that contain live code, equations, visualizations, and narrative text, making it an excellent tool for collaborative AI research and development.<\/p>\n\n\n\nWing<\/a><\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*e56uBfWsHVW4DVilskteLg.png\")
<\/figure>\n\n\n\n![\"\"\/](\"https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*C-2kOGafW6qBieFIWhHJIg.png\")
<\/figure>\n\n\n\nnpm install -g winglang<\/pre>\n\n\n\n
git clone https:\/\/github.com\/winglang\/wing.git<\/pre>\n\n\n\n
npm install<\/pre>\n\n\n\n
wing it<\/pre>\n\n\n\n
2. Vector Data Storage and Analysis Tool<\/h1>\n\n\n\n
SingleStore & Notebooks<\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/miro.medium.com\/v2\/resize:fit:960\/1*f67pAou8tRF7y44nxGvDXw.jpeg\")
<\/figure>\n\n\n\n3. Data Manipulation and Analysis Tool<\/h1>\n\n\n\n
Pandas and NumPy<\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*aYwAe-uo3DaZU7qKZ1-Msw.png\")
<\/figure>\n\n\n\n4. AI & Machine Learning Frameworks<\/h1>\n\n\n\n
TensorFlow and PyTorch<\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*WUA31civiwn4Cxd3Vox6bg.png\")
<\/figure>\n\n\n\nLangChain<\/strong><\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/miro.medium.com\/v2\/resize:fit:1244\/1*qgTxFRveVr0Ry1c6i8M1zQ.png\")
<\/figure>\n\n\n\nLlamaIndex<\/strong><\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/miro.medium.com\/v2\/resize:fit:1280\/1*cEw5o62dErwK0HaO7diEHg.png\")
<\/figure>\n\n\n\n5. Deep Learning Model<\/h1>\n\n\n\n
Keras<\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*HhuuayTnQvND8aZZIrHwow.png\")
<\/figure>\n\n\n\n6. Development and Version Control Platforms<\/h1>\n\n\n\n
GitHub and DVC<\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*fQaKam6TCIHXmc4ZBkbaDw.png\")
<\/figure>\n\n\n\n7. AI Model Deployment and Monitoring<\/h1>\n\n\n\n
Docker and Kubernetes<\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*VRN-qwdP53s1aNS5uBLJPA.png\")
<\/figure>\n\n\n\n8. Cloud Platforms for AI<\/h1>\n\n\n\n
AWS, Google Cloud, and Azure<\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*ITwooQgBmuz0MNgBaZJ_NA.png\")
<\/figure>\n\n\n\n9. Specialized AI Development Tools<\/h1>\n\n\n\n
Jupyter Notebooks<\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/miro.medium.com\/v2\/resize:fit:416\/1*-v25l5tmBUN69lLWCAdzwg.png\")
<\/figure>\n\n\n\nMLflow<\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*ZQ2UK__Wr0ql-gy4MnqMPg.png\")
<\/figure>\n\n\n\n