Tokyo Tech, Tohoku University, Fujitsu and RIKEN launch collaboration to develop large-scale language models: Fujitsu Global

[ad_1]

Tokyo Institute of Technology, Tohoku University, Fujitsu Ltd., RIKEN

Tokyo, May 22, 2023

Tokyo Institute of Technology (Tokyo Tech), Tohoku University, Fujitsu Co., Ltd., and RIKEN have announced that they will begin research and development of distributed training for large language models (LLM). (1) On the supercomputer Fugaku in May 2023 within the scope of the Fugaku Utilization Initiative defined by Japanese policy.

LLMs are deep learning AI models that serve as a core component of generative AI, including ChatGipt. (2). The four organizations will contribute to improving the environment for creating LLMs that can be widely used by academics and companies, improve AI research capabilities in Japan, and share the results to increase the value of using Fugaku in academic and industrial fields. This R&D forward.

Background

While many envision LLMS and generative AI technologies to play a fundamental role in research and development for security, the economy, and society, the development and improvement of these models requires high-performance computing resources that can efficiently process high-performance computing. Data size.

Tokyo Tech, Tohoku University, Fujitsu, and RIKEN are developing LLMS courses that focus on research and development for this purpose.

Application period

May 24, 2023 to March 31, 2024 *Starting period for using fugaku for Japanese policies

The roles of each organization and company

The technology used in this initiative enables the organizations to efficiently perform large-scale language model training on the supercomputer Fugaku’s massively parallel computing environment. The roles of each organization and company are as follows.

Tokyo Institute of Technology: General Process Control, LLMS Parallelism and Acceleration

Tohoku University: Learning data collection, model selection

Fujitsu: LMM

RIKEN: Distributed Parallel and Linkage Acceleration of LLMS, LLMS Acceleration

Future plans

To support Japanese researchers and engineers in developing future LLMs, the four organizations plan to publish research results from initiatives designed to use Fugaku as defined by GitHub policy in Japan. (3) and huggable face (4) In fiscal 2024. In addition, many researchers and engineers are expected to participate in the development of basic models and new applied research, creating efficient methods for further innovative research and business results.

The four firms are from Nagoya University, which develops data generation and learning methods for multimodal applications in industrial fields such as manufacturing, and CyberAgent Inc., which provides data and technology to build LLMs.

Comment from Toshio Endo, Professor, Global Scientific Information and Computing Center, Tokyo Institute of Technology:

“The collaboration combines parallelization and acceleration of large-scale language models using Tokyo Tech and RIKEN’s supercomputer “Fugaku”, Fujitsu’s high-performance computing infrastructure software for Fugaku and performance tuning of AI models, and Tohoku University’s natural language. Technology. In collaboration with Fujitsu in 202X We will use a small research laboratory that we have established under the name of “Fujitsu Collaborative Research Center for Next-Generation Computing Infrastructure”. We look forward to working with our colleagues to contribute to the improvement of Japan’s AI research capabilities by leveraging the deep learning capabilities of “Fugaku”.

Kentaro Inui, a professor at Tohoku University’s Graduate School of Information Science, commented.

“Our goal is to build an open-source, commercially available, and primarily Japanese data-based, comprehensive language model with transparency on the training data. By enabling learning data discovery, this avoids black-box problems, biases, misinformation, and “cheats” common with AI. We expect that it will facilitate robust research to scientifically validate the so-called issues. We will build large-scale models using the insights we gained from the deep learning of Japanese natural language processing at Tohoku University. We look forward to contributing to the development of AI research capacity in our country and beyond, with the results of our research inspired by researchers and developers. By sharing.

Comment from Seishi Okamoto, EVP, Head of Fujitsu Research, Fujitsu Ltd.:

“We are delighted to have the opportunity to leverage the supercomputer Fugaku’s powerful parallel computing resources to advance AI research and expand the research and development of LLMS. Going forward, we aim to incorporate the fruits of this research into Fujitsu’s new AI platform, dubbed “Kozuchi,” to deliver paradigm-shifting applications that contribute to the realization of a sustainable society.

Comment from Satoshi Matsuoka, director of the RIKEN Computational Science Center:

“A64FX (5) The CPU has an AI acceleration function known as SVE.

Software development and optimization are essential to maximize its potential and use for AI applications. We feel that this joint research will play an important role in bringing together LLMs and computer science experts in Japan, RIKEN R-CCS researchers and engineers, to advance the techniques of building LMS on the supercomputer “Fugaku”. Together with our partners, we contribute to making Society 5.0 a reality.

Project name

Distributed Training of Large Language Models on Fugaku (Project Number: hp230254)


About Tokyo Institute of Technology

Tokyo Tech is at the forefront of research and higher education as Japan’s leading university of science and technology. Tokyo Tech researchers excel in fields ranging from materials science to biology, computer science and physics. In the year Founded in 1881, Tokyo Tech hosts more than 10,000 undergraduate and graduate students, who have gone on to become scientific leaders and some of the most sought-after engineers in industry. Tokyo Tech’s community embodies the Japanese philosophy of “monotsukuri,” meaning “technical ingenuity and innovation,” and strives to contribute to society through high-impact research.

https://www.titech.ac.jp/amharic/

About Tohoku University

Tohoku University has 18,000 students in 10 faculties, 15 graduate schools, and six research institutes. About 10 percent of students come from abroad, making it one of the most cosmopolitan academic environments in Japan. Tohoku University’s excellent academic environment, international outlook and research influence have earned it the status of National University designated by the Japanese government in June 2017. It has been ranked first in Times Higher for the past four years. An annual ranking of Japanese universities, highlighting institutional resources, academic quality, and overall student experience.

About Fujitsu

Fujitsu’s mission is to make the world more sustainable by increasing trust in society through innovation. As a digital transformation partner for clients in more than 100 countries, our 124,000 employees work to solve humanity’s greatest challenges. Our services and solutions draw on five key technologies: computing, networks, AI, data and security, and converging technologies, which we bring together to drive sustainable change. Fujitsu Ltd. (TSE:6702) reported consolidated revenue of ¥3.7 trillion (US$28 billion) for the fiscal year ending March 31, 2023, making it Japan’s largest digital services company by market share. Learn more: www.fujitsu.com

About RIKEN

As a leading center for high-performance computing, the RIKEN Center for Computational Science (R-CCS) explores “computer science, by computers, and by computers.” The results of the exploration – technologies such as open source software – are the main competence. R-CCS strives to enhance its core competencies and promote its technologies worldwide. R-CCS partnered with Fujitsu to create Fugaku, the world’s most powerful supercomputer. Full operations of Fugaku are set to begin in March 2021, commanding massive increases in computing capabilities and integration with other IT ecosystems, such as big data and artificial intelligence. He previously operated the R-CCS K computer (2012-2019), which has produced many world-leading science and engineering results not only in academia but also in industry.

Press the contacts

Tokyo Institute of Technology
Department of Public Relations, Tokyo Institute of Technology

Email: media@jim.titech.ac.jp
Phone: + 81-3-5734-2975

Fujitsu Limited
Public and Investor Relations Department

Questions


All company or product names mentioned herein are trademarks or trademarks of their respective owners. The information presented in this press release is correct at the time of publication and is subject to change without prior notice.


day – May 22, 2023

City: Tokyo, Japan.

Company: Tokyo Institute of Technology, Tohoku University, Fujitsu Ltd., RIKEN

[ad_2]

Source link

Leave a Reply

Your email address will not be published. Required fields are marked *