{"id":850,"date":"2023-09-30T06:52:50","date_gmt":"2023-09-30T06:52:50","guid":{"rendered":"https:\/\/tbekk.com\/devstream\/?p=850"},"modified":"2023-10-05T09:44:37","modified_gmt":"2023-10-05T09:44:37","slug":"weekly-ai-and-nlp-news-september-25th-2023","status":"publish","type":"post","link":"https:\/\/tbekk.com\/devstream\/2023\/09\/30\/weekly-ai-and-nlp-news-september-25th-2023\/","title":{"rendered":"Weekly AI and NLP News \u2014 September 25th 2023"},"content":{"rendered":"\n<h2 class=\"wp-block-heading has-medium-gray-color has-text-color\">DALL\u00b7E 3, AlphaMissense, and LoRA for finetuning LLMs for longer context windows<\/h2>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-light-gray-color has-alpha-channel-opacity has-light-gray-background-color has-background is-style-wide\"\/>\n\n\n\n<ul class=\"wp-block-list\">\n<li><em><strong>Link:<\/strong><\/em> <a href=\"https:\/\/medium.com\/nlplanet\/weekly-ai-and-nlp-news-september-18th-2023-3e128fbda17d\"><em>NLPlanet<\/em><\/a><\/li>\n\n\n\n<li><strong><em>Author:<\/em><\/strong> <a href=\"https:\/\/medium.com\/@chiusanofabio94?source=post_page-----3e128fbda17d--------------------------------\"><em>Fabio Chiusano<\/em><\/a><\/li>\n\n\n\n<li><strong><em>Publication Date:<\/em><\/strong> <em>Sept. 25, 2023<\/em><\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-light-gray-color has-alpha-channel-opacity has-light-gray-background-color has-background is-style-wide\"\/>\n\n\n\n<p id=\"076f\">Here are your weekly articles, guides, and news about NLP and AI chosen for you by&nbsp;<a rel=\"noreferrer noopener\" href=\"https:\/\/www.nlplanet.org\/\" target=\"_blank\">NLPlanet<\/a>!<\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"6d7f\">\ud83d\ude0e News From The Web<\/h1>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/openai.com\/dall-e-3\" rel=\"noreferrer noopener\" target=\"_blank\">OpenAI announces DALL\u00b7E 3<\/a>. OpenAI is launching DALL\u00b7E 3, an improved version that excels in following instructions, requires less prompt engineering, and can communicate with ChatGPT. This integration enables users to refine prompts for DALL\u00b7E 3 by describing their ideas to ChatGPT. Starting in October, DALL\u00b7E 3 will be available to ChatGPT Plus and Enterprise customers.<\/li>\n\n\n\n<li><a href=\"https:\/\/www.deepmind.com\/blog\/alphamissense-catalogue-of-genetic-mutations-to-help-pinpoint-the-cause-of-diseases\" rel=\"noreferrer noopener\" target=\"_blank\">DeepMind announces AlphaMissense, a catalogue of genetic mutations to help pinpoint the cause of diseases<\/a>. DeepMind has released AlphaMissense, a model that uses AlphaFold\u2019s protein structure prediction to categorize missense genetic mutations as benign or malign. It surpasses human efforts by classifying 89% of 71 million variants.<\/li>\n\n\n\n<li><a href=\"https:\/\/blogs.microsoft.com\/blog\/2023\/09\/21\/announcing-microsoft-copilot-your-everyday-ai-companion\/\" rel=\"noreferrer noopener\" target=\"_blank\">Announcing Microsoft Copilot, your everyday AI companion<\/a>. Microsoft Copilot will provide tailored assistance based on workplace data and web context. It enhances productivity and creativity in Windows 11, Microsoft 365, Edge, and Bing, while prioritizing privacy. Additionally, Bing and Edge users will enjoy a personalized experience with OpenAI\u2019s DALL.E 3 model, including AI shopping and image creation.<\/li>\n\n\n\n<li><a href=\"https:\/\/blog.google\/products\/bard\/google-bard-new-features-update-sept-2023\/\" rel=\"noreferrer noopener\" target=\"_blank\">Bard can now connect to your Google apps and services<\/a>. The new Bard Extensions feature provides integration with various Google tools, enabling AI professionals to collaborate more effectively. This includes fetching and displaying relevant information from Gmail, Docs, Drive, Maps, YouTube, Flights, and hotels, regardless of its scattered nature.<\/li>\n\n\n\n<li><a href=\"https:\/\/edition.cnn.com\/2023\/09\/23\/us\/fighting-wildfire-with-ai-california-climate\/index.html\" rel=\"noreferrer noopener\" target=\"_blank\">How California is using AI to snuff out wildfires before they explode<\/a>. The California Department of Forestry and Fire Protection is utilizing AI technology to improve wildfire detection and response. Through the Alert California program, AI scans the wilderness for anomalies like smoke, alerting officials when fires are detected.<\/li>\n<\/ul>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"a8b0\">\ud83d\udcda Guides From The Web<\/h1>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/www.adept.ai\/blog\/sherlock-sdc\" rel=\"noreferrer noopener\" target=\"_blank\">Adept.ai shares curious errors occuring during their large training runs<\/a>. Adept.ai, an AI company, shares insights on errors that can occur during large training runs. These errors can lead to learning curve issues, where models may appear fine but not function correctly due to accumulating small errors over time.<\/li>\n\n\n\n<li><a href=\"https:\/\/www.technologyreview.com\/2023\/09\/15\/1079624\/deepmind-inflection-generative-ai-whats-next-mustafa-suleyman\/\" rel=\"noreferrer noopener\" target=\"_blank\">DeepMind\u2019s cofounder: Generative AI is just a phase. What\u2019s next is interactive AI.<\/a>. Mustafa Suleyman, co-founder of DeepMind, emphasizes the positive impact of technology on healthcare and leads an AI policy development team at Google. Backed by influential figures and companies, Suleyman introduces Pi, a friendly AI, and advocates for interactive AI as a means to connect technology with societal impact.<\/li>\n\n\n\n<li><a href=\"https:\/\/ragntune.com\/blog\/gpt3.5-vs-llama2-finetuning\" rel=\"noreferrer noopener\" target=\"_blank\">GPT 3.5 vs Llama 2 fine-tuning: A Comprehensive Comparison<\/a>. A comparison of ChatGPT 3.5 and Llama 2 on an SQL task and a functional representation task reveals that GPT 3.5 slightly outperforms Llama 2. However, the cost of training and deploying GPT 3.5 is 4\u20136 times higher than Llama 2.<\/li>\n\n\n\n<li><a href=\"https:\/\/huggingface.co\/blog\/object-detection-leaderboard\" rel=\"noreferrer noopener\" target=\"_blank\">Object Detection Leaderboard<\/a>. Hugging Face has introduced the Object Detection Leaderboard, featuring top-performing models based on the DETA and DETR architectures.<\/li>\n\n\n\n<li><a href=\"https:\/\/medium.com\/towards-artificial-intelligence\/generative-ai-for-time-series-d808cff2c976\">Generative AI for time-series<\/a>. Generative Adversarial Networks (GANs) offer promise in generating high-quality synthetic time series data, but face challenges in preserving temporal relationships and mapping complex connections. However, a unique architecture called DoppelGANger employs batch generation, auto- normalization, and joint distribution modeling to overcome these challenges and accurately generate realistic temporal patterns.<\/li>\n<\/ul>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"99ce\">\ud83d\udd2c Interesting Papers and Repositories<\/h1>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/arxiv.org\/abs\/2309.12307\" rel=\"noreferrer noopener\" target=\"_blank\">LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models<\/a>. LongLoRA is a method for efficiently extending the context size of pre-trained language models (LLMs) in artificial intelligence. By utilizing sparse local attention during training and dense global attention during inference, this approach allows for cost-effective fine-tuning and maintains performance. LongLoRA demonstrates impressive results on various tasks and enables context extension up to 100k tokens in LLMs.<\/li>\n\n\n\n<li><a href=\"https:\/\/github.com\/vllm-project\/vllm\" rel=\"noreferrer noopener\" target=\"_blank\">vllm-project\/vllm: A high-throughput and memory-efficient inference and serving engine for LLMs<\/a>. vLLM is an open-source serving engine that offers exceptional speed and improved efficiency for LLMs. It integrates seamlessly with Hugging Face, supporting high-throughput serving with advanced decoding algorithms. With its impressive performance, vLLM outperforms Hugging Face Transformers and Text Generation Inference in terms of throughput.<\/li>\n\n\n\n<li><a href=\"https:\/\/arxiv.org\/abs\/2309.11495\" rel=\"noreferrer noopener\" target=\"_blank\">Chain-of-Verification Reduces Hallucination in Large Language Models<\/a>. Chain-of-Verification (CoVe) is a straightforward approach that effectively minimizes hallucinations in Language Model-based systems. Through its systematic process of generating, verifying, and delivering responses, CoVe has proven its success in reducing hallucinations across various tasks, including question answering and text generation.<\/li>\n\n\n\n<li><a href=\"https:\/\/arxiv.org\/abs\/2308.14711\" rel=\"noreferrer noopener\" target=\"_blank\">Fast Feedforward Networks<\/a>. Fast Feedforward Networks (FFF) are binary tree structures with smaller neural networks as leaves, offering significantly faster performance compared to Mixture-of-Experts networks. Despite challenges like fragmentation due to an overly deep tree, FFF networks hold great promise for scenarios requiring fast inference and encoding of minor details.<\/li>\n\n\n\n<li><a href=\"https:\/\/arxiv.org\/abs\/2309.09117\" rel=\"noreferrer noopener\" target=\"_blank\">Contrastive Decoding Improves Reasoning in Large Language Models<\/a>. Contrastive decoding in LLM is a powerful method for reasoning tasks. It surpasses greedy decoding and nucleus sampling, excelling in benchmarks like HellaSwag and GSM8K.<\/li>\n\n\n\n<li><a href=\"https:\/\/arxiv.org\/abs\/2309.08872\" rel=\"noreferrer noopener\" target=\"_blank\">PDFTriage: Question Answering over Long, Structured Documents<\/a>. Researchers have developed PDFTriage, a solution that enhances the performance of Language Model- based question answering systems on structured documents like PDFs. By incorporating document structure and content, PDFTriage outperforms existing models in answering complex questions across various categories.<\/li>\n\n\n\n<li><a href=\"https:\/\/arxiv.org\/abs\/2309.09400\" rel=\"noreferrer noopener\" target=\"_blank\">CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages<\/a>. CulturaX is a curated multilingual dataset containing 6T tokens, designed for language models in 167 languages. The dataset undergoes thorough cleaning stages to ensure top-quality training data for AI language models.<\/li>\n\n\n\n<li><a href=\"https:\/\/arxiv.org\/abs\/2309.09958\" rel=\"noreferrer noopener\" target=\"_blank\">An Empirical Study of Scaling Instruct-Tuned Large Multimodal Models<\/a>. Researchers have found that improving image resolution and mixing multimodal-language data during training can enhance the performance of multimodal models like LLaVA and MiniGPT-4. Additionally, they have discovered that tuning visual instructions can further improve the language capabilities of these models.<\/li>\n\n\n\n<li><a href=\"https:\/\/arxiv.org\/abs\/2309.08532\" rel=\"noreferrer noopener\" target=\"_blank\">Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers<\/a>. EvoPrompt, a new framework using evolutionary algorithms, optimizes prompt generation for language models like GPT-3.5 and Alpaca. It surpasses human-engineered prompts and current methods, demonstrating its effectiveness for language tasks.<\/li>\n\n\n\n<li><a href=\"https:\/\/arxiv.org\/abs\/2309.08520\" rel=\"noreferrer noopener\" target=\"_blank\">Scaling Laws for Sparsely-Connected Foundation Models<\/a>. Researchers have discovered a unique scaling law that shows the relationship between weight sparsity, non-zero parameters, and training data volume in foundation models. They also found that the optimal sparsity level for performance increases with more data.<\/li>\n<\/ul>\n\n\n\n<p id=\"1e54\">Thank you for reading! If you want to learn more about NLP, remember to follow&nbsp;<a href=\"https:\/\/www.nlplanet.org\/\" rel=\"noreferrer noopener\" target=\"_blank\">NLPlanet<\/a>. You can find us on&nbsp;<a href=\"https:\/\/www.linkedin.com\/company\/nlplanet\" rel=\"noreferrer noopener\" target=\"_blank\">LinkedIn<\/a>,&nbsp;<a href=\"https:\/\/twitter.com\/nlplanet_\" rel=\"noreferrer noopener\" target=\"_blank\">Twitter<\/a>,&nbsp;<a href=\"https:\/\/medium.com\/nlplanet\">Medium<\/a>, and our&nbsp;<a href=\"https:\/\/discord.gg\/zfC862H2dJ\" rel=\"noreferrer noopener\" target=\"_blank\">Discord server<\/a>!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>DALL\u00b7E 3, AlphaMissense, and LoRA for finetuning LLMs for longer context windows Here are your weekly articles, guides, and news about NLP and AI chosen for you by&nbsp;NLPlanet! \ud83d\ude0e News&#8230; <a class=\"read-more-link\" href=\"https:\/\/tbekk.com\/devstream\/2023\/09\/30\/weekly-ai-and-nlp-news-september-25th-2023\/\">Read more &raquo;<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[181,51,115,19,201,132,9],"tags":[27,312,310,215,311,20,302,40],"class_list":["post-850","post","type-post","status-publish","format-standard","hentry","category-ai-2","category-article","category-data-science","category-ml","category-nlp","category-nn","category-news","tag-ai","tag-alphamissense","tag-dalle-e-3","tag-llm","tag-lora","tag-ml","tag-news","tag-nlp"],"_links":{"self":[{"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/posts\/850","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/comments?post=850"}],"version-history":[{"count":3,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/posts\/850\/revisions"}],"predecessor-version":[{"id":862,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/posts\/850\/revisions\/862"}],"wp:attachment":[{"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/media?parent=850"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/categories?post=850"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/tags?post=850"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}