Season 1 - Episode 05: Language independence and material properties
This week we are joined by Sebastian Ruder. He is a research scientist at DeepMind, London. He has also worked at a variety of institutions such as AYLIEN, Microsoft, IBM's Extreme Blue, Google Summer of Code, and SAP. These experiences were completed in tangent with his studies which included studying Computational Linguistics at the University of Heidelberg, Germany and at Trinity College, Dublin before undertaking a PhD in Natural Language Processing and Deep Learning at the Insight Research Centre for Data Analytics.
This week we discuss language independence and diversity in natural language processing whilst also taking a look at the attempts to identify material properties from images.
As discussed in the podcast if you would like to donate to the current campaign of "CREATE DONATE EDUCATE" which supports Stop Hate UK then please find the link below:
https://www.shorturl.at/glmsz
Please also find additional links to help support black colleagues in the area of research;
Black in AI twitter account: https://twitter.com/black_in_ai
Mentoring and proofreading sign-up to support our Black colleagues in research: https://twitter.com/le_roux_nicolas/status/1267896907621433344?s=20
Underrated ML Twitter: https://twitter.com/underrated_ml
Sebastian Ruder Twitter: https://twitter.com/seb_ruder
Please let us know who you thought presented the most underrated paper in the form below: https://forms.gle/97MgHvTkXgdB41TC8
Links to the papers:
“On Achieving and Evaluating Language-Independence in NLP” - https://journals.linguisticsociety.org/elanguage/lilt/article/view/2624.html
"The State and Fate of Linguistic Diversity and Inclusion in the NLP World” - https://arxiv.org/abs/2004.09095
"Recognizing Material Properties from Images" - https://arxiv.org/pdf/1801.03127.pdf
Additional Links:
Student perspectives on applying to NLP PhD programs: https://blog.nelsonliu.me/2019/10/24/student-perspectives-on-applying-to-nlp-phd-programs/
Tim Dettmer's post on how to pick your grad school: https://timdettmers.com/2020/03/10/how-to-pick-your-grad-school/
Rachel Thomas' blog post on why you should blog: https://medium.com/@racheltho/why-you-yes-you-should-blog-7d2544ac1045
Emily Bender's The Gradient article: https://thegradient.pub/the-benderrule-on-naming-the-languages-we-study-and-why-it-matters/
Paper on order-sensitive vs order-free methods: https://www.aclweb.org/anthology/N19-1253.pdf
"Exploring the Origins and Prevalence of Texture Bias in Convolutional Neural Networks": https://arxiv.org/abs/1911.09071
Sebastian's website where you can find all his blog posts: https://ruder.io/
