Logo

Medical Intelligence Society Summit 2021 - Shared screen with speaker view
Fatme Charafeddine
56:32
Thank you so much! I would be honoured to :)
Flora Wan
56:33
Happy Birthday Bob!!
Zaidon Al-Falahi
56:40
Happy Birthday Bob!
Fatme Charafeddine
56:45
Happy belated Birthday Bob :)
Robert Hoyt
57:46
I would let you know how old I am but I don’t have the carbon dating results back from the Smithsonian
Pele Yu
57:48
Happy birthday Bob!!!!
Pele Yu
59:29
Will sign up for MIS!!
Iman Attackpour
01:00:41
Happy birthday BOB
Anthony Chang
01:03:35
AI in healthcare is in great hands with our younger generation getting more involved!
Anthony Chang
01:07:46
Happy Birthday Bob and we are grateful for all your wisdom and generosity. You are an amazing role model for all of us.
Anthony Chang
01:08:53
A warm welcome to all of you from all over the world and thank you for spending the morning/afternoon/evening with us.
Anthony Chang
01:10:20
Good data science without clinical relevance or impact is just good data science.
Anthony Chang
01:11:08
Good data science + Clinical relevance or impact = Great AI in medicine project
Panida Piboolnurak
01:11:55
If there is a clinical office/hospital model with AI embedded in all aspects of care and work, then anyone can visit and see how it works and how good it is. Is there such a model we can visit?
Scott Campbell MD, MPH
01:14:37
Great recent article by Charles Martin here in SF re the train and test problem https://www.nature.com/articles/s41467-021-24025-8
John Lee
01:16:08
I think "machine learning" tends to make clinicians think of something unfamiliar. If we couch it as similar to what we've done for decades but with more data and better algorithms, it will resonate more.
Robert Hoyt
01:16:21
RCTs will require funding so in the meantime we need and should demand external validation
John Lee
01:16:34
This is very much what I think we want to accomplish as a society.
Arlen Meyers
01:16:55
Integrating data science concepts into clinical rotations in medical schools is a way to education and train them. But, how if the clinical faculty are not educated or trained?
Robert Hoyt
01:17:00
Great talk. Kudos for your grasp of a complicated subject
Irene Schanberger
01:21:55
Using Ornge
Zaidon Al-Falahi
01:22:14
Arlen, I've been talking closely with one of the deans of medicine who is now very pro AI. That medical school is part of a university with a strong engineering faculty, including professors with dual medical and engineering degrees. The main issue being faced? How to cram in more material into an already very full curriculum. What some universities are going to do is offer optional AI literacy courses
Scott Campbell MD, MPH
01:22:26
How about having one part of morning report being dedicated to how Ai/ML could be leveraged to predict the problem at hand, such as “classifiers for predicting admission” or “Med Sure length of stay, etc”?
Arlen Meyers
01:23:11
Some ideas:
Anthony Chang
01:23:23
If you are interested in a two day comprehensive LIVE course with an amazing core and guest faculty, the American Board of AI in Medicine (ABAIM) has a monthly review course especially designed for clinicians with no or little AI background and also well designed for data scientists as well. ABAIM.org
Arlen Meyers
01:23:30
1. AI ethics conference
Arlen Meyers
01:23:40
2. Data science grand rounds
Arlen Meyers
01:23:52
3. AI M/M conferences
Arlen Meyers
01:24:05
4. AIntrepreneurship fellowship for medical students
Arlen Meyers
01:24:37
5. Medical student incubator in collabaration with engineering and business faculty
Hatim Abdulhussein
01:24:48
@zaidon @arlen that's a great approach but I think the key for us is to show the value of data and machine learning initiatives embedded into all aspects of curriculum already covered - and point to its value when those topics are covered - a much more integrated approach!
Arlen Meyers
01:25:44
6. Intro "lunch and learn" sessions with visiting community domain expert faculty. I'm teaching this 10 week course to first year medical students at Colorado starting in few weeks.
Hatim Abdulhussein
01:25:50
and then optional add ons for those that want more through further academic opportunities
Hatim Abdulhussein
01:27:18
at Brunel University London we will be creating a longitudinal curriculum and covering AI and machine learning in a TBL session in year 1 and 2, followed by hackathon type initiatives in years 4/5. And have optional student selected modules available in year 3
Avneesh Khare
01:27:41
Good evening Everyone!
Zaidon Al-Falahi
01:27:51
@Hatim, I agree, but I think at this stage the issue is not to show value, the promise is clear and the scientific plausibility is well demonstrated. The issue is, how to integrate it into the clinical workflow in a sound manner while addressing the issues Arlen mentioned. That is the problem, I think!
Zaidon Al-Falahi
01:28:11
Thanks Arlen, noted!
Panida Piboolnurak
01:28:46
Great ideas. These will be for medical students level. How about fellowship in AI in medicine after completing residency program?
Scott Campbell MD, MPH
01:29:14
Bob, what are your thoughts on Orange vs something like MS Azure ML studio which one can also get a free account and seems pretty good for the projects I have been running?
Arlen Meyers
01:29:39
It would be useful to create a place where healthcare professional school educators can share best practices, curriculum, offerings and outcomes. Perhaps a regularly scheduled office hours on MIS platform or something like that?
Hatim Abdulhussein
01:29:52
agree @zaidon if we can't get that clinical buy in its hard to get educational buy in. I was watching patch adams and I guess the same problem faced compassion, empathy and key patient doctor communication skills when that first came through into education and training
Arlen Meyers
01:30:22
I;m at arlen.meyers@ucdenver.edu and LinkedIn if you want to compare notes.
Arlen Meyers
01:30:46
You get educational buy in when the accreditation criteria demand it.
Scott Campbell MD, MPH
01:32:08
I love the notion of getting to medical students but my recent experience has been a larger appetite for Ai among RESIDENCY and FELLOWSHIP directors for a number of reasons
DrDoRo®Institute
01:32:11
Besides TensorFlow are there any other open-source AI's ; especially suitable for medical students to get their "feet wet" and "play & learn in a sandbox"?
Arlen Meyers
01:34:38
@Scott Campbell I agree that we need a buy incontinuum from premed to med to residency to fellowship. Not sure residency program directors are a receptive audience at this point but certainly a group that needs more education and engagement.
John Michael FINLEY
01:36:55
Bob - great presentation. Where does one locate the recordings of the ORANGE workshops?
Peter Chang
01:36:58
Great discussion above. We created a one-year AI fellowship in Radiology here at UCI, we have our first fellow this year.
Peter Chang
01:37:54
I will present an overview of Tensorflow/Keras at 9 AM (in 10 min). I can also talk about Pytorch if there is interest!
Anthony Chang
01:38:53
https://abaim.org/
Peter Chang
01:39:15
This is my CS 190 class I teach to CS undergraduate students (does require a strong working knowledge of Python but no data science experience): https://github.com/peterchang77/dl_tutor/tree/master/cs190
Anthony Chang
01:41:54
Great job moderating Hatim and thank you Anant and Bob for such a good session on making it real.
Anthony Chang
01:42:27
Thanks Peter for being here every year to share with us a session on medical image and AI.
Peter Chang
01:42:31
This will the 9 AM session if folks want to follow-up along with the Tensorflow/Keras classifier for CXR pneumonia detection in Google Colab: https://bit.ly/2D9ZBrX
Hatim Abdulhussein
01:43:56
@scott @arlen @peter its really great to hear what you are doing. Here in the UK I am trying to embed AI opportunities into our postgraduate training - recently the London AI Centre has proposed a clinical AI education programme which we hope to pilot and evaluate! Still trying to get our postgraduate deans to agree to it
Flora Wan
01:46:32
Hi Peter, would be great if you could briefly mention how Tensorflow/Keras compares to PyTorch as I’ve learned mainly TF and wonder what are the differences between TF and PyTorch and in what types of scenarios you would choose one over the other. Thanks so much!
Anthony Chang
01:48:18
Great AI music!
Peter Chang
01:48:19
Of course, I can talk about it at the start of the 9 AM PT session, but from a data science perspective, there are both very good and mature libraries that will allow you to build just about any model you want.
Flora Wan
01:48:51
Thanks, Peter!
Anthony Chang
01:49:15
The ABAIM will be working on a fellowship program on AI in medicine in the very near future (later this year). Stay in touch!
Avneesh Khare
01:49:37
Great news Dr Chang
Avneesh Khare
01:49:41
:)
Panida Piboolnurak
01:49:54
That's great!!
Fatme Charafeddine
01:49:54
So exciting indeed!!!
Anthony Chang
01:50:31
I look at ABAIM and its certification and review course as well as fellowship and board certification as US models that can transfer to other countries. A real life transfer learning model!
Avneesh Khare
01:51:20
Swarm intelligence ✌️
Hatim Abdulhussein
01:51:26
absolutely Anthony and its so exciting to be starting our collaboration journey with ABAIM here in England
Fatme Charafeddine
01:52:53
A model with good representation :)
Flora Wan
01:55:54
Thanks for the great summary, Peter, that’s very helpful!
Anthony Chang
02:00:58
It will be such an honor to work with the UK and HEE on this as a tribute to Alan Turing and his pioneering work. He would be proud of all of us!
Anthony Chang
02:02:08
Thanks to our friend Zaidon, we will be in Australia later this year! Hoping some day to do the course in person!
Zaidon Al-Falahi
02:02:58
Thanks Anthony, so looking forward to it!
Zaidon Al-Falahi
02:03:15
We need this all over the world!
DrDoRo®Institute
02:14:24
@Anthony ...have you tried Endel [https://endel.io] , which creates personalized , AI powered music adaptable to an individual’s immediate conditions to reduce stress, improve sleep, and boost productivity?
Anthony Chang
02:18:36
No but looks amazing! Thanks for the intel!
Anthony Chang
02:19:02
I may open the AIMed Global Summit with it!
Anthony Chang
02:23:51
Great job Peter in explaining a difficult subject!
Animesh Tandon, M.D., M.S.
02:24:08
If layer 7 is looking for pneumonia at every 64th pixel, what if the 64th pixel is not right where the pneumonia is?
DrDoRo®Institute
02:29:36
@Arlene Meyers, yes , I agree, it would be useful to create a place where healthcare professional school educators can share best practices, curriculum, offerings and outcomes. Count me in and let me know.
Animesh Tandon, M.D., M.S.
02:38:01
Is that because the layers below are feeding up into it?
Flora Wan
02:38:46
How would you suggest to visualize what part of the image the model is looking at to make the prediction to confirm that it’s predicting based on what we would expect from a clinical point of view vs an unexpected pattern in the image dataset?
Zaidon Al-Falahi
02:39:40
Is a heat map applicable here to figure out what the ~15% is looking at?
Flora Wan
02:40:54
Thank you!
Zaidon Al-Falahi
02:42:35
Awesome, thanks!
Anthony Chang
02:43:08
Thank you Peter for a great introduction to CNN.
Avneesh Khare
02:43:25
Awesome dissecto
Avneesh Khare
02:43:53
Awesome dissection of a CNN. Felt like sitting in the anatomy dissection lab.
Pele Yu
02:44:03
Agree, it was very informative and understanding step wise explanation of layers.
Animesh Tandon, M.D., M.S.
02:44:17
This was great, thanks
Pele Yu
02:44:53
@Bob - can we do a Tensor vs Orange comparison?
Zaidon Al-Falahi
02:45:10
I guess an important question is the feasibility of combining the outcome of CXR prediction with say, clinical variables to predict the likelihood of COVID? Not sure if ensemble applies strictly here, but in the real world, we rarely use just one modality to make a diagnosis unless it is very specific
Ajay Perumbeti
02:46:47
are there standards for deployment as you start thinking about EMR databases, interfaces, and web applications. Maybe everyone develops their own way. We are startomg to build and testing models, but are a bit lost for next steps you mentioned
Hanna Lu
02:46:54
@ Dr. Chang, based on your experience, is there any effective way to address the overfitting problems when the dataset is relatively small?
Zaidon Al-Falahi
02:47:55
That is fantastic, thank you!
Anthony Chang
02:49:23
One of the best dividends about spending time with CNN and images is that you learn a lot about how you look at medical images and how you may want to modify your perspective.
Flora Wan
02:49:35
Thank you Peter for such a clear and insightful session, you explained this better in an hour than my deep learning instructor could do in a whole semester! :)
Ben Babu
02:49:59
what can we do to improve model accuracy and external clinical validation ?
Anthony Chang
02:50:58
Thank you Peter and Hatim for a great hour!
Peter Chang
02:51:04
@Ajay — there are essentially no AI/ML specific standards currently, which is a significant bottleneck to deployment. What exists currently builds on top of standard FHIR (EHR) or DICOM-based (radiology) standards, but these legacy informatics systems do not address any of the AI/mL specific needs.
Peter Chang
02:52:12
There are some companies now interested in the pure deployment as a platform model (e.g., hosting models produced by other companies or researchers). There is some potential here, but the problem is that there are still quite a few to choose from, without any clear consensus or standards between the companies.
Peter Chang
02:53:56
@Hanna yes — managing overfitting is the single most common thing we must work with in medical problems since data is so scarce. The key techniques to think about include: keeping your model as COMPACT as possible (see MobileNet or DenseNet for some good state-of-the-art solutions); extensive data augmentation; regularization methods including L2 regularization and batch normalization; dropout.
Peter Chang
02:54:24
Also in general, the more detailed your algorithm makes a prediction, the less data you need. For example an object localization or segmentation model requires LESS data than a standard global classifier.
Peter Chang
02:55:01
@Flora — much thanks for your kind comments. Next time I teach my course at UCI I will let you know; you are more than welcome to audit remotely.
Flora Wan
02:55:24
That would be awesome! :)
Peter Chang
02:57:25
@Ben — generalization of models in the context of clinical deployment is very tricky and a major area of research currently. In general, we need to curate larger, for diverse datasets from as many different hospitals as possible to ensure the algorithm can handle edges cases or rare diseases. Additionally there are interesting new techniques such as distributed or federated learning which allow models to learn simultaneously from datasets that might be located at different hospitals (this is one of my current areas of research interest).
Ben Babu
02:58:29
excellent ! thanks for the insight 👍
Zaidon Al-Falahi
03:01:35
I wonder when do we get an EMR that automatically integrates such algorithms, with an ability to tweak and control what goes into each prediction. Or is that asking too much?
Anthony Chang
03:03:19
Multimodal AI that will blend EHR and genomics will be next in radiomics
Robert Hoyt
03:05:47
There needs to be universal data representation. I hope Google Healthcare API that represents EHR data as a FHIR standard will be in the right direction but that might be too optimistic
Anthony Chang
03:07:03
Thank you Louis and Hanna for coming on a Saturday to discuss NLP.
Louis Ehwerhemuepha
03:08:16
https://github.com/choc-mi3/mis_nlp_notebook
Louis Ehwerhemuepha
03:20:28
She is removing the header information that is common to all notes
Zaidon Al-Falahi
03:22:38
Can you predict the DRG/ICD codes this way?
Pele Yu
03:22:46
Is there a way to determine the repeating words?
Dony Ang
03:22:55
how did you get the corpus ? Was it scanned from OCR ?
Peter Chang
03:24:46
For complex NLP problems, consider transformers like the state-of-the-art GPT family of models.
Peter Chang
03:25:24
(If you Google “GPT-3 examples” you will find some amazing tools based off of this algorithm)
Hanna Lu
03:25:44
https://portal.dbmi.hms.harvard.edu/projects/n2c2-nlp/
Joe Morgan
03:25:56
Has GPT-3 been made open by "Open"AI yet?
Zaidon Al-Falahi
03:26:59
NLP is kind of the future in medicine, yet significant concerns arise with using pt notes due to identifiable information. I wonder if the future means that every hospital needs to run its own algorithms in house, or if there's another way to federate the learning somehow
Pele Yu
03:28:20
I like “Its the humans fault…” :-)
Anthony Chang
03:29:59
We have to own the limitations sometimes…
Panida Piboolnurak
03:31:10
We may need "Doctor Language Processing" (DLP)...
Anthony Chang
03:35:19
Great term Panida!
Panida Piboolnurak
03:35:28
Thank you.
Anthony Chang
03:35:31
NLP may have a tough time!
Dony Ang
03:35:32
I thought there are several cloud managed services that offer this DLP. I know Azure has one called Azure Text Analytics for health that has been trained against several healthcare corpuses so that it can recognize ICD-10 etc etc. And of course, both AWS and Google Cloud have similar services.
Anthony Chang
03:35:54
There is also ClinicalBERT.
Joe Morgan
03:36:16
also several transformers models building upon Bert
Anthony Chang
03:36:38
https://github.com/EmilyAlsentzer/clinicalBERT
Dony Ang
03:36:39
And bioBERT and BlueBERT too …. Have you guys used it in real project(s) ?
Anthony Chang
03:37:03
https://aclanthology.org/W19-1909/
Hatim Abdulhussein
03:37:21
i agree @zaidon but here in UK I worry if all hospitals will be able to have the skills and infrastructure to make good use of their in house data
Joe Morgan
03:37:33
sorry to nag, but is GPT3 still something that requires a private key/permission from openAI (or whomever)? I have wanted to experiment with it.
Joe Morgan
03:38:56
if anyone aware of above it would be much appreciated
Joe Morgan
03:40:01
Thank you! As someone who is in private practice it has been hard to find.
Joe Morgan
03:40:21
hard to access*
Anthony Chang
03:40:37
ELMO, BERT, and BigBIRD as well as OSCAR for NLP. A Sesame Street full house.
Joe Morgan
03:41:11
haha lots of NLP pun fun out there
Flora Wan
03:41:39
Would GPT-3 be used more for text generation or summarization though vs classification like all the BERT models?
Anthony Chang
03:43:51
I think all of the above as there is very little current experience. Like CNN, we will find a myriad of applications with GPT3 and other NLP transformer type tools. Exciting times when AI in medicine has a bit of a dip…
Anthony Chang
03:44:03
Takeaways;
Flora Wan
03:46:38
In this example, would you say that CNN works well because we’re looking for specific keyword related to the medical condition vs needing to understand the sequential context of the language itself which might require something like RNN?
Anthony Chang
03:46:41
Takeaways: CONVERSATIONAL: We need more clinicians who can be conversational with AI in clinical medicine to be able to be more interactive with our data science colleagues or tools. CLINICAL: We need much more clinical relevance and impact with our AI in medicine projects and papers and this schism is getting bigger (personal opinion). COLLABORATE: Humans need to work even closer together to innovate and find a way to navigate the difficult milieu of data and IT infrastructure.
Panida Piboolnurak
03:46:42
If we still do not know the diagnosis at the discharge, and we put in as DDx list, how the NLP can get the data from it.
DrDoRo®Institute
03:47:11
Thank you @ Hanna, impressive work !
Anthony Chang
03:47:53
http://proceedings.mlr.press/v68/suresh17a.html
Flora Wan
03:48:37
Thank you!
Avneesh Khare
03:49:05
:D
Panida Piboolnurak
03:50:38
Thank you. Great talk!!
Pele Yu
03:50:39
Thank Hannah!
Fatme Charafeddine
03:51:03
Thank you so much! Wonderful sessions
Flora Wan
03:51:40
Fabulous sessions over these two days, can’t wait to see what we’ll be talking about a year from now!
JAWAHAR JAGARAPU
03:52:43
Thanks so much for a wonderful conference
Avneesh Khare
03:52:45
GPT3 will be talking to us next year ;)
Avneesh Khare
03:53:04
Had a great learning experience over these 2 days
Anthony Chang
03:53:42
Please check out our society journal Intelligence Based Medicine
Avneesh Khare
03:53:45
Great talks. Thank you MIS and everyone behind this summit and this wonderful society.
Scott Campbell MD, MPH
03:53:58
As a pandemicologist…this group is one of the single most important groups in the world to learn from what happened the past 18 mos and save more lives in the next one…
Joe Morgan
03:54:04
Great couple of days! Thanks to all who made it happen.
Shingirai Kagodora
03:54:11
Thank you very much its been great 2days. Was watching from South Africa. Looking forward to collaboration.
Avneesh Khare
03:54:14
For keeping excitement alive in medicine!\
Zaidon Al-Falahi
03:54:15
Thank you everyone for making these meetings possible and for sharing knowledge and experiences
Scott Campbell MD, MPH
03:54:43
Tom: Go to med school, not Goldman Sachs!!!
Brendan Dunphy
03:54:47
well said @tom!
Tom Murickan
03:56:46
Thank you all! Medical school has always been the plan, so apologies to Goldman-Sachs, but I'll be taking my skills and engagement elsewhere :)
Avneesh Khare
03:57:36
I think we all should get Neuralink implants and then do swarm learning :D
Scott Campbell MD, MPH
03:58:06
Thanks
Panida Piboolnurak
03:58:07
Bye!
DrDoRo®Institute
03:58:10
Much appreciated . Thank y'all !