AI Newsletter #005 (2023/06/19 – 2023/06/25)

Big Company

Microsoft: Released Orca, A 13B Model Like GPT-4
Microsoft has released a new AI model called Orca, which is a 13 billion language model that imitate and learn from large language models like GPT-4. It has already surpassed Vicuna by 100% on complex zero-shot reasoning benchmarks, and is 42% faster than conventional AI models on AGIEval. Orca is said to be on par with ChatGPT on benchmarks like BBH, and has demonstrated competitive performance on academic examinations such as SAT, LSAT, GRE, and GMAT.

Google: Predicting Heart Disease Through Eye Scans

Sundar Pichai, CEO of Google, has announced a major shift in the health tech industry, utilizing Google’s AI technologies. Google’s AI can predict cardiovascular events through an eye scan, and is expected to be approved for independent operation soon. It can also detect other diseases such as dementia, multiple sclerosis, Parkinson’s, Alzheimer’s, and schizophrenia. Google’s AI was able to distinguish between the retinal images of two patients 70% of the time, signaling a shift towards a new AI-powered paradigm for medical discovery.

IBM: Expand Partnership with Adobe To Deliver Content Supply Chain Solution Using Generative AI
IBM and Adobe have announced an expansion of their partnership to help brands accelerate their content supply chains with next-generation AI. IBM Consulting is launching a portfolio of Adobe consulting services to help clients build an integrated content supply chain ecosystem that improves collaboration, creativity, speed, automation, and visibility.

Cisco: Launches New AI Networking Chips
Cisco Systems has released two new networking chips, G200 and G202, designed to improve AI and machine learning performance. These chips are expected to reduce the number of switches needed by 40%, reduce lag, and be more power efficient than competing products from Broadcom and Marvell Technology.

DeepMind: Unveils Self-Improving ‘RoboCat’ that Uses AI to Teach Robots without Supervision

DeepMind’s RoboCat is an AI-powered robot that can learn and solve problems without human supervision. It can adapt its self-improvement training to transition from two-fingered to three-fingered robot arms, and can learn household tasks in just 25 minutes. This technology marks significant progress towards building general-purpose robots that can perform everyday tasks.

New Product & Product Updates

Midjourney: Realising Version 5.2’s Zoom Out Feature
Midjourney has released version 5.2 of its AI-powered image synthesis model, which includes a new “zoom out” feature. It allows users to enlarge images without distorting resolution or aspect ratio, as well as mini-tools for experimenting with aesthetics, color palettes, and structural arrangements.

Consensus: Research Search Engine
When you need to access professional information, searching through literature can often be a daunting task.Consensus uses AI to power you search experience. Its data is taken entirely from research papers and can display the basic content of articles. It will be a valuable tool for scientific research, learning, content creation, among others. More importantly, it is 100% free.


Emerging Architectures for LLM Applications by A16Z
“In this post, we’re sharing a reference architecture for the emerging LLM app stack. It shows the most common systems, tools, and design patterns we’ve seen used by AI startups and sophisticated tech companies. This stack is still very early and may change substantially as the underlying technology advances, but we hope it will be a useful reference for developers working with LLMs now.”

Frontiers, Startups, 2023++ by Shyamal Anadkat
Shyamal Anadkat is an applied AI engineer at OpenAI. In this blog, he shares his thoughts about recent developments in key verticals and the factors that determine the success of startups in this space.

Research of the Week

Chip-Chat: Challenges and Opportunities in Conversational Hardware Design
Through a simple English conversation, researchers from the NYU Tandon School of Engineering used GPT-4 to create a chip.

Specifically, GPT-4 generated a viable Verilog through back-and-forth dialogue. The benchmark test and processor were then sent to the Skywater 130 nm shuttle machine for successful tapeout.

As shown in the figure, Verilog, a very important part of chip design and manufacturing code, was generated by researchers using GPT-4 through prompt words.

Textbooks Are All You Need
The researchers trained a 1.3B (1.3 billion parameter) model called phi-1 using a dataset that is only 7B tokens in size but is of “textbook quality”. Phi-1 focuses on the task of coding, specifically on writing Python functions and their corresponding documentation strings. The training data was composed of online-filtered textbook quality data of 6B tokens and textbook and exercise data generated by GPT-3.5 of 1B tokens. During training, the model was trained on 70 billion tokens for 8 passes. It was then fine-tuned on data of less than 200 million tokens. The entire training process took place on 8 A100s and lasted for 4 days.

The results show that despite being several orders of magnitude smaller in dataset and model size than competitor models, phi-1 achieved a 50.6% accuracy on pass@1 in HumanEval and 55.5% in MBPP, which is one of the best self-reported numbers generated by a single LLM.

Want to receive TechNavi Newsletter in email?