[vc_row][vc_column][vc_headings linewidth=”0″ borderwidth=”1″ borderclr=”#000000″ title=”Falcon LLM” google_fonts=”font_family:Comfortaa%3A300%2Cregular%2C700|font_style:700%20bold%20regular%3A700%3Anormal” titlesize=”60″ titleclr=”#000000″]Open-Source LLMs[/vc_headings][vc_single_image image=”4240″ img_size=”full” alignment=”center”][vc_column_text]Falcon LLM is a large language model (LLM) with 40 billion parameters that can generate natural language and code. It was developed by Technology Innovation Institute (TII) in Abu Dhabi and is open-sourced under the Apache License Version 2.0. Falcon LLM can be used for various tasks such as chatbots, content creation, language translation, and sentiment analysis. It is one of the most efficient and powerful LLMs in the world, outperforming GPT-3 and other state-of-the-art models on several benchmarks.[/vc_column_text][vc_separator][/vc_column][/vc_row][vc_row][vc_column][vc_headings style=”theme4″ borderclr=”#000000″ style2=”image” title=”How It Works” google_fonts=”font_family:Comfortaa%3A300%2Cregular%2C700|font_style:700%20bold%20regular%3A700%3Anormal” lineheight=”3″ titlesize=”40″ titleclr=”#000000″ image_id=”2854″][/vc_headings][vc_column_text]Falcon is a massive decoder-only model with 40 billion parameters that can generate text autoregressively. It was trained on 1 trillion tokens using 384 GPUs on AWS for two months.
The pretraining data for Falcon came from public web crawls that were filtered, deduplicated, and cleaned to remove machine-generated text and adult content. The resulting pretraining dataset had almost five trillion tokens.
This dataset was supplemented with selected sources such as academic papers and social media conversations to diversify Falcon AI skills.
Finally, Falconâs performance was evaluated on open-source benchmarks such as EAI Harness, HELM, and BigBench.[/vc_column_text][vc_separator][/vc_column][/vc_row][vc_row][vc_column][vc_headings style=”theme4″ borderclr=”#000000″ style2=”image” title=”Falcon AI Family” google_fonts=”font_family:Comfortaa%3A300%2Cregular%2C700|font_style:700%20bold%20regular%3A700%3Anormal” lineheight=”3″ titlesize=”40″ titleclr=”#000000″ image_id=”4240″]
The below table provides a comprehensive overview of the artefacts in the Falcon LLM family:
[/vc_headings][vc_column_text css=”.vc_custom_1686384722791{border-top-width: 2px !important;border-right-width: 2px !important;border-bottom-width: 2px !important;border-left-width: 2px !important;background-color: #b5b5b5 !important;border-left-color: #0a0a0a !important;border-left-style: solid !important;border-right-color: #0a0a0a !important;border-right-style: solid !important;border-top-color: #0a0a0a !important;border-top-style: solid !important;border-bottom-color: #0a0a0a !important;border-bottom-style: solid !important;border-radius: 1px !important;}”]
| Artefact | Link | Type | Details |
|---|---|---|---|
| đ„Â Falcon-40B | Here | pretrained model | 40B parameters trained on 1,000 billion tokens. |
| Falcon-40B-Instruct | Here | instruction/chat model | Falcon-40B finetuned on the Baize dataset. |
| đ„Â Falcon-7B | Here | pretrained model | 6.7B parameters trained on 1,500 billion tokens. |
| Falcon-7B-Instruct | Here | instruction/chat model | Falcon-7B finetuned on the Baize, GPT4All, and GPTeacher datasets. |
| đ RefinedWeb | Here | pretraining web dataset | ~600 billion “high-quality” tokens. |
| Falcon-RW-1B | Here | pretrained model | 1.3B parameters trained on 350 billion tokens. |
| Falcon-RW-7B | Here | pretrained model | 7.5B parameters trained on 350 billion tokens. |
[/vc_column_text][vc_separator][/vc_column][/vc_row][vc_row][vc_column][vc_headings style=”theme4″ borderclr=”#000000″ style2=”image” title=”Falcon AI Use Cases” google_fonts=”font_family:Comfortaa%3A300%2Cregular%2C700|font_style:700%20bold%20regular%3A700%3Anormal” lineheight=”3″ titlesize=”40″ titleclr=”#000000″ image_id=”2871″][/vc_headings][evc_icon_with_text type=”icon-left” icon_library=”fontawesome” title_tag=”” icon_fontawesome=”fas fa-file-alt” title=”Creative text and problem solving” text=”Falcon can generate original and engaging text for various purposes, such as marketing, storytelling, education, and entertainment. It can also help users solve complex problems by providing logical and actionable suggestions.” text_color=”#0a0000″][evc_icon_with_text type=”icon-left” icon_library=”fontawesome” title_tag=”” icon_fontawesome=”fas fa-robot” title=”Chatbots and customer service” text=”Falcon can power chatbots that can understand and communicate fluently in different languages, provide relevant and helpful information, and handle diverse and complex requests from customers.” text_color=”#0a0000″][evc_icon_with_text type=”icon-left” icon_library=”fontawesome” title_tag=”” icon_fontawesome=”fas fa-file-signature” title=”Content generation and sentiment analysis” text=”Falcon LLM can produce high-quality content for websites, blogs, social media, and other platforms, tailored to the target audience and tone. It can also analyze the sentiment of the text and provide feedback and suggestions to improve it.” text_color=”#0a0000″][evc_icon_with_text type=”icon-left” icon_library=”fontawesome” title_tag=”” icon_fontawesome=”fas fa-cogs” title=”Process automation and efficiency” text=”Falcon LLM can help Emirati companies and startups reduce and automate repetitive and tedious tasks, such as data entry, report writing, email management, and scheduling. This will allow employees to focus on more creative and strategic work.” text_color=”#0a0000″][evc_icon_with_text type=”icon-left” icon_library=”fontawesome” title_tag=”” icon_fontawesome=”fas fa-user-check” title=”Personal assistant and lifestyle” text=”Falcon AI can assist users in their daily lives by providing reminders, recommendations, tips, and fun facts. It can also help users with their hobbies, interests, goals, and well-being.” text_color=”#0a0000″][vc_separator color=”sandy_brown” border_width=”3″][/vc_column][/vc_row][vc_row][vc_column width=”1/3″][vc_btn title=”Official Website” color=”warning” align=”center” i_align=”right” i_icon_fontawesome=”fas fa-external-link-alt” add_icon=”true” link=”url:https%3A%2F%2Ffalconllm.tii.ae%2F|target:_blank”][/vc_column][vc_column width=”1/3″][vc_btn title=”Project Page” color=”inverse” align=”center” i_align=”right” i_icon_fontawesome=”fas fa-external-link-alt” add_icon=”true” link=”url:https%3A%2F%2Fhuggingface.co%2Ftiiuae|target:_blank”][/vc_column][vc_column width=”1/3″][vc_btn title=”Dataset Page” color=”pink” align=”center” i_align=”right” i_icon_fontawesome=”fas fa-external-link-alt” add_icon=”true” link=”url:https%3A%2F%2Fhuggingface.co%2Fdatasets%2Ftiiuae%2Ffalcon-refinedweb|target:_blank”][/vc_column][/vc_row][vc_row][vc_column][vc_separator][vc_headings style=”theme4″ borderclr=”#000000″ style2=”image” title=”FAQ” google_fonts=”font_family:Comfortaa%3A300%2Cregular%2C700|font_style:700%20bold%20regular%3A700%3Anormal” lineheight=”1.5″ titlesize=”40″ titleclr=”#000000″ image_id=”2875″][/vc_headings][vc_separator][vc_toggle title=”Is Falcon LLM free?” color=”juicy_pink”]Yes, Falcon LLM is free for research and commercial use under the Apache License Version 2.0.[/vc_toggle][vc_separator color=”mulled_wine” border_width=”2″][vc_toggle title=”How can I access Falcon LLM?” color=”juicy_pink”]You can access Falcon LLM through the Hugging Face Page[/vc_toggle][vc_separator color=”mulled_wine” border_width=”2″][vc_toggle title=”What is Falcon-40B?” color=”juicy_pink”]Falcon-40B is a general-purpose large language model with 40 billion parameters trained on one trillion tokens of text and code. It is one of the four versions of Falcon LLM available.[/vc_toggle][vc_separator color=”mulled_wine” border_width=”2″][vc_toggle title=”Can I submit my case proposal?” color=”juicy_pink”]Yes, you can submit your case proposal for using Falcon LLM to the Technology Innovation Institute (TII) through their website. TII is offering training compute power and commercialization opportunities for exceptional use cases.[/vc_toggle][vc_separator color=”mulled_wine” border_width=”2″][/vc_column][/vc_row]
0 Comments