[ad_1]
Tokyo Institute of Technology, Tohoku University, Fujitsu Ltd., RIKEN
Tokyo, May 22, 2023
Tokyo Institute of Technology (Tokyo Tech), Tohoku University, Fujitsu Co., Ltd., and RIKEN have announced that they will begin research and development of distributed training for large language models (LLM). (1) On the supercomputer Fugaku in May 2023 within the scope of the Fugaku Utilization Initiative defined by Japanese policy.
LLMs are deep learning AI models that serve as a core component of generative AI, including ChatGipt. (2). The four organizations will contribute to improving the environment for creating LLMs that can be widely used by academics and companies, improve AI research capabilities in Japan, and share the results to increase the value of using Fugaku in academic and industrial fields. This R&D forward.
Background
While many envision LLMS and generative AI technologies to play a fundamental role in research and development for security, the economy, and society, the development and improvement of these models requires high-performance computing resources that can efficiently process high-performance computing. Data size.
Tokyo Tech, Tohoku University, Fujitsu, and RIKEN are developing LLMS courses that focus on research and development for this purpose.
Application period
May 24, 2023 to March 31, 2024 *Starting period for using fugaku for Japanese policies
The roles of each organization and company
The technology used in this initiative enables the organizations to efficiently perform large-scale language model training on the supercomputer Fugaku’s massively parallel computing environment. The roles of each organization and company are as follows.
Tokyo Institute of Technology: General Process Control, LLMS Parallelism and Acceleration
Tohoku University: Learning data collection, model selection
Fujitsu: LMM
RIKEN: Distributed Parallel and Linkage Acceleration of LLMS, LLMS Acceleration
Future plans
To support Japanese researchers and engineers in developing future LLMs, the four organizations plan to publish research results from initiatives designed to use Fugaku as defined by GitHub policy in Japan. (3) and huggable face (4) In fiscal 2024. In addition, many researchers and engineers are expected to participate in the development of basic models and new applied research, creating efficient methods for further innovative research and business results.
The four firms are from Nagoya University, which develops data generation and learning methods for multimodal applications in industrial fields such as manufacturing, and CyberAgent Inc., which provides data and technology to build LLMs.
Comment from Toshio Endo, Professor, Global Scientific Information and Computing Center, Tokyo Institute of Technology:
“The collaboration combines parallelization and acceleration of large-scale language models using Tokyo Tech and RIKEN’s supercomputer “Fugaku”, Fujitsu’s high-performance computing infrastructure software for Fugaku and performance tuning of AI models, and Tohoku University’s natural language. Technology. In collaboration with Fujitsu in 202X We will use a small research laboratory that we have established under the name of “Fujitsu Collaborative Research Center for Next-Generation Computing Infrastructure”. We look forward to working with our colleagues to contribute to the improvement of Japan’s AI research capabilities by leveraging the deep learning capabilities of “Fugaku”.
Kentaro Inui, a professor at Tohoku University’s Graduate School of Information Science, commented.
“Our goal is to build an open-source, commercially available, and primarily Japanese data-based, comprehensive language model with transparency on the training data. By enabling learning data discovery, this avoids black-box problems, biases, misinformation, and “cheats” common with AI. We expect that it will facilitate robust research to scientifically validate the so-called issues. We will build large-scale models using the insights we gained from the deep learning of Japanese natural language processing at Tohoku University. We look forward to contributing to the development of AI research capacity in our country and beyond, with the results of our research inspired by researchers and developers. By sharing.
Comment from Seishi Okamoto, EVP, Head of Fujitsu Research, Fujitsu Ltd.:
“We are delighted to have the opportunity to leverage the supercomputer Fugaku’s powerful parallel computing resources to advance AI research and expand the research and development of LLMS. Going forward, we aim to incorporate the fruits of this research into Fujitsu’s new AI platform, dubbed “Kozuchi,” to deliver paradigm-shifting applications that contribute to the realization of a sustainable society.
Comment from Satoshi Matsuoka, director of the RIKEN Computational Science Center:
“A64FX (5) The CPU has an AI acceleration function known as SVE.
Software development and optimization are essential to maximize its potential and use for AI applications. We feel that this joint research will play an important role in bringing together LLMs and computer science experts in Japan, RIKEN R-CCS researchers and engineers, to advance the techniques of building LMS on the supercomputer “Fugaku”. Together with our partners, we contribute to making Society 5.0 a reality.
Project name
Distributed Training of Large Language Models on Fugaku (Project Number: hp230254)
About Tokyo Institute of Technology
Tokyo Tech is at the forefront of research and higher education as Japan’s leading university of science and technology. Tokyo Tech researchers excel in fields ranging from materials science to biology, computer science and physics. In the year Founded in 1881, Tokyo Tech hosts more than 10,000 undergraduate and graduate students, who have gone on to become scientific leaders and some of the most sought-after engineers in industry. Tokyo Tech’s community embodies the Japanese philosophy of “monotsukuri,” meaning “technical ingenuity and innovation,” and strives to contribute to society through high-impact research.
https://www.titech.ac.jp/amharic/
About Tohoku University
Tohoku University has 18,000 students in 10 faculties, 15 graduate schools, and six research institutes. About 10 percent of students come from abroad, making it one of the most cosmopolitan academic environments in Japan. Tohoku University’s excellent academic environment, international outlook and research influence have earned it the status of National University designated by the Japanese government in June 2017. It has been ranked first in Times Higher for the past four years. An annual ranking of Japanese universities, highlighting institutional resources, academic quality, and overall student experience.
About Fujitsu
Fujitsu’s mission is to make the world more sustainable by increasing trust in society through innovation. As a digital transformation partner for clients in more than 100 countries, our 124,000 employees work to solve humanity’s greatest challenges. Our services and solutions draw on five key technologies: computing, networks, AI, data and security, and converging technologies, which we bring together to drive sustainable change. Fujitsu Ltd. (TSE:6702) reported consolidated revenue of ¥3.7 trillion (US$28 billion) for the fiscal year ending March 31, 2023, making it Japan’s largest digital services company by market share. Learn more: www.fujitsu.com
About RIKEN
As a leading center for high-performance computing, the RIKEN Center for Computational Science (R-CCS) explores “computer science, by computers, and by computers.” The results of the exploration – technologies such as open source software – are the main competence. R-CCS strives to enhance its core competencies and promote its technologies worldwide. R-CCS partnered with Fujitsu to create Fugaku, the world’s most powerful supercomputer. Full operations of Fugaku are set to begin in March 2021, commanding massive increases in computing capabilities and integration with other IT ecosystems, such as big data and artificial intelligence. He previously operated the R-CCS K computer (2012-2019), which has produced many world-leading science and engineering results not only in academia but also in industry.
Press the contacts
Tokyo Institute of Technology
Department of Public Relations, Tokyo Institute of Technology
Email: media@jim.titech.ac.jp
Phone: + 81-3-5734-2975
Fujitsu Limited
Public and Investor Relations Department
Questions
All company or product names mentioned herein are trademarks or trademarks of their respective owners. The information presented in this press release is correct at the time of publication and is subject to change without prior notice.
day – May 22, 2023
City: Tokyo, Japan.
Company: Tokyo Institute of Technology, Tohoku University, Fujitsu Ltd., RIKEN
[ad_2]
Source link