AI Passes U.S. Medical Licensing Exam

— Two papers show that large language models, including ChatGPT, can pass the USMLE

by Michael DePeau-Wilson, Enterprise & Investigative Writer, MedPage  January 19, 2023

Two artificial intelligence (AI) programs — including ChatGPT — have passed the U.S. Medical Licensing Examination (USMLE), according to two recent papers.

The papers highlighted different approaches to using large language models to take the USMLE, which is comprised of three exams: Step 1, Step 2 CK, and Step 3.

ChatGPTopens in a new tab or window is an artificial intelligence (AI) search tool that mimics long-form writing based on prompts from human users. It was developed by OpenAI, and became popular after several social media posts showed potential uses for the tool in clinical practice, often with mixed resultsopens in a new tab or window.

The first paper, published on medRxivopens in a new tab or window in December, investigated ChatGPT’s performance on the USMLE without any special training or reinforcement prior to the exams. According to Victor Tseng, MD, of Ansible Health in Mountain View, California, and colleagues, the results showed “new and surprising evidence” that this AI tool was up to the challenge.

Tseng and team noted that ChatGPT was able to perform at >50% accuracy across all of the exams, and even achieved 60% in most of their analyses. While the USMLE passing threshold does vary between years, the authors said that passing is approximately 60% most years.

“ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement,” they wrote, noting that the tool was able to demonstrate “a high level of concordance and insight in its explanations.”

“These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making,” they concluded.

The second paper, published on arXivopens in a new tab or window, also in December, evaluated the performance of another large language model, Flan-PaLM, on the USMLE. The key difference between the two models was that this model was heavily modified to prepare for the exams, using a collection of medical question-answering databases called the MultiMedQA, explained Vivek Natarajan, an AI researcher, and colleagues.

Flan-PaLM achieved 67.6% accuracy in answering the USMLE questions, which was about 17 percentage points higher than the previous best performance conducted using PubMed GPT.

Natarajan and team concluded that large language models “present a significant opportunity to rethink the development of medical AI and make it easier, safer and more equitable to use.”

ChatGPT, along with other AI programs, have been showing up as the subject — and sometimes as the co-author — of new research papers focused on testing the technology’s usefulness in medicine.

Of course, healthcare professionals have also expressed concerns over these developments, especially when ChatGPT is being listed as an author on research papers. A recent article from Natureopens in a new tab or window highlighted the uneasiness from would-be colleagues and co-authors of the emerging technology.

One objection to the use of AI programs in research was based on whether they can be truly capable of making meaningful scholarly contributions to a paper, while another objection emphasized that AI tools can’t consent to be a co-author in the first place.

The editor of one of the papersopens in a new tab or window that listed ChatGPT as an author said it was an error that would be corrected, according to the Nature article. Still, researchers have published several papers now touting these AI programs as useful tools in medical education, research, and even clinical decision making.

Natarajan and colleagues concluded in their paper that large language models could become a beneficial tool in medicine, but their first hope was that their findings would “spark further conversations and collaborations between patients, consumers, AI researchers, clinicians, social scientists, ethicists, policymakers and other interested people in order to responsibly translate these early research findings to improve healthcare.”

Michael DePeau-Wilson is a reporter on MedPage Today’s enterprise & investigative team. He covers psychiatry, long covid, and infectious diseases, among other relevant U.S. clinical news. Follow 

Primary Source

medRxiv

Source Reference: opens in a new tab or windowKung TH, et al “Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models” medRxiv 2022; DOI: 10.1101/2022.12.19.22283643.

Secondary Source

arXiv

Source Reference: opens in a new tab or windowSinghal K, et al “Large language models encode clinical knowledge” arXiv 2022; DOI: 10.48550/arXiv.2212.13138.

Photo/Draper is a well-known venture capitalist in Silicon Valley, USA, with a net worth of more than US$1 billion. Photo by Chen Zhijun – Translated then republished from https://www.gvm.com.tw/article/98167

This article is from the 2023/01 issue “

Tim Draper is a well-known venture capitalist in Silicon Valley, and his investment projects include Skype, Space X, and Tesla. In an exclusive interview with “Foresight”, he talked about the attitudes of central banks to virtual cryptocurrencies, mentioned how to assist Taiwan’s start-ups, and also expressed his thoughts on Chinese President Xi Jinping.

Jian Jiahong 

On the morning of the interview, the “Foresight” team walked into the hotel’s business center meeting room. At the end of the long table sat a smiling Draper, and his partners and assistants were scattered in the meeting room. We quickly stepped forward to shake hands and took a closer look. Draper, who is about 200 centimeters tall, was wearing a tie with a Bitcoin ( BitCoin ) pattern on his chest, which was full of financial flavor. His business card is more eye-catching. The QR Code printed on it shows that this business card is worth one ten-thousandth of a unit of Bitcoin. 

Draper said “Ni Hao” in Chinese, and the smile on his face resolved the slightly tense atmosphere of the first meeting between the two parties. The following is a summary of the interview: 

“Foresight Magazine” asked (hereinafter referred to as the question): How do you see the future of cryptocurrency and the attitude that central banks should have? 

Draper’s answer (hereinafter referred to as A): After the bankruptcy of FTX , the world’s top three cryptocurrency exchanges , people have slowly accepted ” decentralized currencies “, which are more important than ” centralized currencies ” . Flexible assumptions. I have always advocated the decentralized currency system. FTX is a centralized exchange. Centralization may lead to wrong development. Just like the central bank faces difficulties and makes decisions on the currency. I think it is wrong for the central bank to try to control the currency. 

The government must trust your people and be free for economic value to grow. South Korea is a perfect example. Compared with the total control of the North Korean government, South Korea is completely liberalized. The per capita income of South Korea is 460 times that of North Korea. It is estimated that in seven years, the average height of South Koreans will be higher than the average height of North Koreans. 4 inches (about 10 cm) taller . The people of South Korea eat better, live better, and live longer because they have a government that trusts the people and creates a free environment. 

So, I recommend that all governments trust their people and allow innovation, and therefore set up the “Draper innovation index ( DII ), this index compares the entrepreneurial environment around the world, and the United States still ranks first, as do the Nordic countries Great place to start a business, right now, there is global competition trying to attract talent, business, entrepreneurs, capital. (Editor’s note: Taiwan’s DII ranks 15th as of publication ) 

Suggestion for central bank to allow bitcoin to be listed as fiat currency 

Most of the countries ranked in front of the DII are free and trust the people, and expect prosperous economic growth. These countries are the environment that attracts business investment. If the country is controlled by the government, it will lower the morale of the people and hurt the economic growth. People end up poor and hungry because they don’t have the freedom to create their lives. 

Therefore, I would suggest that the centralized central banks of various countries try to make Bitcoin ( Bitcoin ) one of the legal tenders. At present, I cannot shop with NT dollars in the US retail market, but I can use Bitcoin transactions. I think such a thing will More and more, when people get food, clothing, and shelter through Bitcoin, they gradually no longer need centralized currency. 

It takes time, any new technology is adopted, the popularity of the public will ebb and flow, but I think eventually it will be bigger than the Internet, I remember how people commented on the Internet at the beginning, people used to think that using It is not polite to send a letter by email, and it is not considered to be able to use a credit card on the Internet. Many people do not trust the Internet. 

But the internet is a big part of society today, and we’re seeing something similar now in terms of cryptocurrencies, and I think in two to ten years, Bitcoin will be a very big part of society, although FTX had a big crisis and hit confidence, however, FTX is a centralized exchange, and people do need to do their homework before switching to cryptocurrencies. 

Q: How do you view China’s future and the impact of Xi Jinping’s continued power on the world situation? 

A: I would be happy to see China return to a free world system. I think when China opens up its markets and the world can do business with China, there is almost no barrier between the United States and China. It feels like we are all part of this world, as if One country. 

This is why I chose to come to Taiwan. 

We interviewed all Taiwanese entrepreneurs who have invested, looking for potential investors, and specially produced the ” Meet The Drapers ” reality program in Taiwan to discover outstanding entrepreneurs. If China is willing to open its market in the future, I would still like to go to China, but , now Taiwan is a free country, so I am here. 

Furthermore, I don’t think there is a need to be nervous about geopolitics , leaders don’t need to start wars. Again, this is the problem with centralized government control, allowing the world to function more freely would reduce geopolitical tension, and frankly, this is a (geopolitical) tension between two leaders, not me problems with peoples of other countries. I like my Chinese friends, but when it comes to geopolitical tensions, it’s like two people deciding whether they like or hate each other, and we are innocently involved. 

Think about what Russian President Vladimir Putin is doing, he is destroying people’s homes, killing lives, and his decisions affect the entire country and the world suffers from it. This is not a complicated decision. 

The job of a leader is to set people free 

The best leaders in the world are on a mission to set people free. For example, George Washington in the United States, Mikhail Gorbachev in Russia , Deng Xiaoping in China, and Lee Kuan Yew in Singapore are all willing to give freedom to the people, let them create life, and let the beautiful Things happen, and it’s this kind of freedom that allows for amazing economic growth. 

I always remember the scene I saw in China: I was driving on the only asphalt road in Beijing, staying in the only international hotel in Beijing, surrounded by people who exchanged rice with beans in hats, some people were riding bicycles, some people were walking or wandering around At that time, the quality of life of Chinese people was not good, but after the implementation of Deng Xiaoping’s policy of “letting some people get rich first”, China began to promote a liberalized market. The world has witnessed that China’s economy has grown rapidly for 40 years. 

I think Xi Jinping can learn a lesson from the very different development of the Korean peninsula, that instead of keeping all Chinese people closed, the people need to be free. 

Q: Please talk about the investment environment in Taiwan

A: I think the advantage of Taiwanese is that they are well educated. 

Taiwan is a democratic and free regime, with good semiconductors and equipment, and a long history of manufacturing. At the same time, Taiwan is located in a good geographical location in Asia, and it is easy to trade with Taiwan in all directions. I have seen many interesting things in Taiwan. The company, filled with many innovative ideas, overall, Taiwan is a good place. 

Q: What do you think are the promising areas of development in the future? 

Answer: I have noticed that some countries have begun to seek to use Bitcoin as a national legal tender, which can attract entrepreneurs and funds. 

For example, El Salvador , one of the poorest countries in the world, tries to attract the best entrepreneurs and bring in a lot of investment capital. What I think the government should know is that they are competing with the whole world, in a 220 country environment, with more competitors emerging every day. 

Another example is Estonia ( Estonia ), they created a virtual residency program (or e-residency plan), the country currently has a population of 1.4 million, but the number of virtual residents (or e-residents) also has 300,000 . The significance of this is that Estonia began to imagine that governance could be managed virtually. The government should be a kind of insurance for the people, and what the people need is guaranteed in terms of safety, health, education, etc. 

When everything is virtualized online, we don’t need to subject everything to big, heavy government control. Many government-provided projects (in a virtual network) can be provided better, cheaper, and better. 

I find that the Taiwanese government is actively exercising trust externally because it wants to unbundle some of the technology to the private sector, which I think is beneficial. 

Q: How do you see the future of Taiwanese media and the challenges facing Chinese media? 

A: Freedom of speech in Taiwan’s media may be the most effective way to influence China’s 1.2 billion people. 

I think freedom of speech is really important, and it is even more important for the media, because you have to be able to say what you hear and hear freely, and you have to be able to express your thoughts and opinions freely. Interestingly, the Internet can change this situation, or say that Internet media has already changed the media ecology in a way that I did not expect. A story can be spread to all parts of the world through the Internet. 

I also expect that there will be decentralized media on the Internet, full of interesting stories everywhere, and always bring out interesting things. The way media operates is more diverse than I expected. Although it is not obvious at present, but This is possible in China.  

Draper Timothy Cook Draper  
Born: 1958  
Education: Harvard Business School  
Experience: Three generations of grandparents are venture capitalists, founded Draper Association, and established Draper University, Draper Venture Capital Network, Draper Innovation index 

For more exciting reports, welcome to join the official account of Vision Line !

On the understandings specified below, the Office of the United States Attorney for the Southern District of New York (“this Office”) will accept a guilty plea from Caroline Ellison (“the defendant”) to the above-referenced Superseding Information (the “Information”).

Former Alameda Research CEO could be released on $250,000 bail, but she’s not allowed to leave the United States.

The plea deal between Ellison and the Office of the United States Attorney for the Southern District of New York was published on Dec. 21. According to the document, the former Alameda exec will be spared of all major charges, which could have seen her sentenced to up to 110 years in prison.

Ellison was accused of seven counts. Two accused her of committing wire fraud on customers of FTX and engaging and conspiring to do so. Another two alleged she committed wire fraud on the lenders of Alameda Research and conspired to do so. Count five charged her with conspiracy to commit commodities fraud, and count six alleged conspiracy to commit securities fraud on FTX’s equity investors. The seventh count accused her of conspiring to commit money laundering.

The Attorney’s Office agreed not to prosecute Ellison on any of those seven counts in exchange for her cooperation — the complete disclosure of all the information and documents demanded by prosecutors.