Alex Graves (computer scientist)

Alex Graves
Alex Graves
Alma mater	University of Edinburgh (BSc); Technical University of Munich (PhD);
Known for	Connectionist temporal classification; Neural Turing machine; Differentiable neural computer;
	Scientific career
Fields	Artificial Intelligence; Recurrent neural networks; Handwriting recognition; Speech recognition;
Institutions	DeepMind; University of Toronto; Dalle Molle Institute for Artificial Intelligence Research
Thesis	Supervised sequence labelling with recurrent neural networks (2008)
Doctoral advisor	Jürgen Schmidhuber
Website	www.cs.toronto.edu/~graves
	Show more

Alex Graves is a computer scientist.^[1]

Education

Graves earned his Bachelor of Science degree in Theoretical Physics from the University of Edinburgh^[when?] and a PhD in artificial intelligence from the Technical University of Munich supervised by Jürgen Schmidhuber at the Dalle Molle Institute for Artificial Intelligence Research.^[2]^[3]

Career and research

After his PhD, Graves was postdoc working with Schmidhuber at the Technical University of Munich and Geoffrey Hinton^[4] at the University of Toronto.

At the Dalle Molle Institute for Artificial Intelligence Research, Graves trained long short-term memory (LSTM) neural networks by a novel method called connectionist temporal classification (CTC).^[5] This method outperformed traditional speech recognition models in certain applications.^[6] In 2009, his CTC-trained LSTM was the first recurrent neural network (RNN) to win pattern recognition contests, winning several competitions in connected handwriting recognition.^[7]^[8] Google uses CTC-trained LSTM for speech recognition on the smartphone.^[9]^[10]

Graves is also the creator of neural Turing machines^[11] and the closely related differentiable neural computer.^[12]^[13] In 2023, he wrote the paper Bayesian Flow Networks.^[14]

References

^ ^a ^b Alex Graves publications indexed by Google Scholar
^ Graves, Alex (2008). Supervised sequence labelling with recurrent neural networks (PDF) (PhD thesis). Technischen Universitat Munchen. OCLC 1184353689.
^ "Alex Graves". Canadian Institute for Advanced Research. Archived from the original on 1 May 2015.
^ "Marginally Interesting: What is going on with DeepMind and Google?". Blog.mikiobraun.de. 28 January 2014. Retrieved May 17, 2016.
^ Alex Graves, Santiago Fernandez, Faustino Gomez, and Jürgen Schmidhuber (2006). Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural nets. Proceedings of ICML’06, pp. 369–376.
^ Fernández, Santiago; Graves, Alex; Schmidhuber, Jürgen (2007). "An Application of Recurrent Neural Networks to Discriminative Keyword Spotting". Artificial Neural Networks – ICANN 2007. Lecture Notes in Computer Science. Vol. 4669. pp. 220–229. doi:10.1007/978-3-540-74695-9_23. ISBN 978-3-540-74693-5.
^ Graves, Alex; and Schmidhuber, Jürgen; Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks, in Bengio, Yoshua; Schuurmans, Dale; Lafferty, John; Williams, Chris K. I.; and Culotta, Aron (eds.), Advances in Neural Information Processing Systems 22 (NIPS'22), December 7th–10th, 2009, Vancouver, BC, Neural Information Processing Systems (NIPS) Foundation, 2009, pp. 545–552 https://dl.acm.org/doi/10.5555/2981780.2981848
^ Graves, A.; Liwicki, M.; Fernandez, S.; Bertolami, R.; Bunke, H.; Schmidhuber, J. (2009). "A Novel Connectionist System for Unconstrained Handwriting Recognition". IEEE Transactions on Pattern Analysis and Machine Intelligence. 31 (5): 855–868. Bibcode:2009ITPAM..31..855G. doi:10.1109/TPAMI.2008.137. PMID 19299860.
^ Google Research Blog. The neural networks behind Google Voice transcription. August 11, 2015. By Françoise Beaufays http://googleresearch.blogspot.co.at/2015/08/the-neural-networks-behind-google-voice.html
^ Google Research Blog. Google voice search: faster and more accurate. September 24, 2015. By Haşim Sak, Andrew Senior, Kanishka Rao, Françoise Beaufays and Johan Schalkwyk – Google Speech Team http://googleresearch.blogspot.co.uk/2015/09/google-voice-search-faster-and-more.html
^ "Google's Secretive DeepMind Startup Unveils a "Neural Turing Machine"". Retrieved May 17, 2016.
^ Graves, Alex; Wayne, Greg; Reynolds, Malcolm; Harley, Tim; Danihelka, Ivo; Grabska-Barwińska, Agnieszka; Colmenarejo, Sergio Gómez; Grefenstette, Edward; Ramalho, Tiago (2016-10-12). "Hybrid computing using a neural network with dynamic external memory". Nature. 538 (7626): 471–476. Bibcode:2016Natur.538..471G. doi:10.1038/nature20101. ISSN 1476-4687. PMID 27732574. S2CID 205251479.
^ "Differentiable neural computers | DeepMind". DeepMind. 12 October 2016. Retrieved 2016-10-19.
^ Graves, Alex; Rupesh Kumar Srivastava; Atkinson, Timothy; Gomez, Faustino (2023). "Bayesian Flow Networks". arXiv:2308.07037 [cs.LG].

[gs-1] Alex Graves publications indexed by Google Scholar

[phd-2] Graves, Alex (2008). Supervised sequence labelling with recurrent neural networks (PDF) (PhD thesis). Technischen Universitat Munchen. OCLC 1184353689.

[3] "Alex Graves". Canadian Institute for Advanced Research. Archived from the original on 1 May 2015.

[MyUser_Blog.mikiobraun.de_May_17_2016c-4] "Marginally Interesting: What is going on with DeepMind and Google?". Blog.mikiobraun.de. 28 January 2014. Retrieved May 17, 2016.

[graves2006-5] Alex Graves, Santiago Fernandez, Faustino Gomez, and Jürgen Schmidhuber (2006). Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural nets. Proceedings of ICML’06, pp. 369–376.

[fernandez2007keyword-6] Fernández, Santiago; Graves, Alex; Schmidhuber, Jürgen (2007). "An Application of Recurrent Neural Networks to Discriminative Keyword Spotting". Artificial Neural Networks – ICANN 2007. Lecture Notes in Computer Science. Vol. 4669. pp. 220–229. doi:10.1007/978-3-540-74695-9_23. ISBN 978-3-540-74693-5.

[7] Graves, Alex; and Schmidhuber, Jürgen; Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks, in Bengio, Yoshua; Schuurmans, Dale; Lafferty, John; Williams, Chris K. I.; and Culotta, Aron (eds.), Advances in Neural Information Processing Systems 22 (NIPS'22), December 7th–10th, 2009, Vancouver, BC, Neural Information Processing Systems (NIPS) Foundation, 2009, pp. 545–552 https://dl.acm.org/doi/10.5555/2981780.2981848

[8] Graves, A.; Liwicki, M.; Fernandez, S.; Bertolami, R.; Bunke, H.; Schmidhuber, J. (2009). "A Novel Connectionist System for Unconstrained Handwriting Recognition". IEEE Transactions on Pattern Analysis and Machine Intelligence. 31 (5): 855–868. Bibcode:2009ITPAM..31..855G. doi:10.1109/TPAMI.2008.137. PMID 19299860.

[GoogleVoiceTranscription-9] Google Research Blog. The neural networks behind Google Voice transcription. August 11, 2015. By Françoise Beaufays http://googleresearch.blogspot.co.at/2015/08/the-neural-networks-behind-google-voice.html

[GoogleVoiceSearch-10] Google Research Blog. Google voice search: faster and more accurate. September 24, 2015. By Haşim Sak, Andrew Senior, Kanishka Rao, Françoise Beaufays and Johan Schalkwyk – Google Speech Team http://googleresearch.blogspot.co.uk/2015/09/google-voice-search-faster-and-more.html

[MyUser_Https:_May_17_2016c-11] "Google's Secretive DeepMind Startup Unveils a "Neural Turing Machine"". Retrieved May 17, 2016.

[DNCnature2016-12] Graves, Alex; Wayne, Greg; Reynolds, Malcolm; Harley, Tim; Danihelka, Ivo; Grabska-Barwińska, Agnieszka; Colmenarejo, Sergio Gómez; Grefenstette, Edward; Ramalho, Tiago (2016-10-12). "Hybrid computing using a neural network with dynamic external memory". Nature. 538 (7626): 471–476. Bibcode:2016Natur.538..471G. doi:10.1038/nature20101. ISSN 1476-4687. PMID 27732574. S2CID 205251479.

[13] "Differentiable neural computers | DeepMind". DeepMind. 12 October 2016. Retrieved 2016-10-19.

[14] Graves, Alex; Rupesh Kumar Srivastava; Atkinson, Timothy; Gomez, Faustino (2023). "Bayesian Flow Networks". arXiv:2308.07037 [cs.LG].

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

Authority control databases
International	ISNI VIAF WorldCat
National	Germany United States Czech Republic Korea
Academics	Association for Computing Machinery Scopus zbMATH Google Scholar DBLP MathSciNet
People	DDB
Other	IdRef

Alex Graves (computer scientist)

Alex Graves (computer scientist)

Alex Graves (computer scientist)

Add your contribution

Hub overview

Root Wikipedia Article

Alex Graves (computer scientist)

Key Information

Education

Career and research

References

Alex Graves
Alma mater	University of Edinburgh (BSc) Technical University of Munich (PhD)
Known for	Connectionist temporal classification Neural Turing machine Differentiable neural computer
Scientific career
Fields	Artificial Intelligence Recurrent neural networks Handwriting recognition Speech recognition^[1]
Institutions	DeepMind University of Toronto Dalle Molle Institute for Artificial Intelligence Research
Thesis	Supervised sequence labelling with recurrent neural networks (2008)
Doctoral advisor	Jürgen Schmidhuber

Website	www.cs.toronto.edu/~graves