1、! ! AI INDEX, NOVEMBER 2017!1! ! AI INDEX, NOVEMBER 2017 STEERING COMMITTEE Yoav Shoham (chair) Stanford University Raymond Perrault SRI International Erik Brynjolfsson MIT Jack Clark OpenAI PROJECT MANAGER Calvin LeGassick !2! ! AI INDEX, NOVEMBER 2017 TABLE OF CONTENTS Introduction to the AI Index
2、 2017 Annual Report 5 Overview 7 Volume of Activity 9 Academia 9 Published Papers 9 Course Enrollment 11 Conference Attendance 14 Industry 16 AI-Related Startups 16 AI-Related Startup Funding 17 Job Openings 18 Robot Imports 21 Open Source Software 23 GitHub Project Statistics 23 Public Interest 25
3、Sentiment of Media Coverage 25 Technical Performance 26 Vision 26 Object Detection 26 Visual Question Answering 27 Natural Language Understanding 28 Parsing 28 Machine Translation 29 Question Answering 30 Speech Recognition 31 !3! ! AI INDEX, NOVEMBER 2017 Theorem Proving 32 SAT Solving 33 Derivativ
4、e Measures 34 Towards Human-Level Performance? 37 Whats Missing? 41 Expert Forum 44 Get Involved! 68 Acknowledgements 70 Appendix A: Data Description for example, how well computers can understand images and prove mathematical theorems. The methodology used to collect each data set is detailed in th
5、e appendix. These first two sets of data confirm what is already well recognized: all graphs are “up and to the right,” reflecting the increased activity in AI e#orts as well as the progress of the technology. In the Derivative Measures section we investigate the relationship between trends. We also
6、 introduce an exploratory measure, the AI Vibrancy Index, that combines trends across academia and industry to quantify the liveliness of AI as a field. When measuring the performance of AI systems, it is natural to look for comparisons to human performance. In the Towards Human-Level Performance se
7、ction we outline a short list of notable areas where AI systems have made significant progress towards !7! ! AI INDEX, NOVEMBER 2017 matching or exceeding human performance. We also discuss the di#iculties of such comparisons and introduce the appropriate caveats. Discussion Sections Following the d
8、isplay of the collected data, we include some discussion of the trends this report highlights and important areas this report entirely omits. Part of this discussion centers on the limitations of the report. This report is biased towards US-centric data sources and may overestimate progress in techn
9、ical areas by only tracking well-defined benchmarks. It also lacks demographic breakdowns of data and contains no information about AI Research & Development investments by governments and corporations. These areas are deeply important and we intend to tackle them in future reports. We further discu
10、ss these limitations and others in the Whats Missing section of the report. As the reports limitations illustrate, the AI Index will always paint a partial picture. For this reason, we include subjective commentary from a cross-section of AI experts. This Expert Forum helps animate the story behind
11、the data in the report and adds interpretation the report lacks. Finally, where the experts dialogue ends, your opportunity to Get Involved begins. We will need the feedback and participation of a larger community to address the issues identified in this report, uncover issues we have omitted, and b
12、uild a productive process for tracking activity and progress in Artificial Intelligence. !8! ! AI INDEX, NOVEMBER 2017 VOLUME OF ACTIVITY Academia Published Papers view more information in appendix A1 The number of Computer Science papers published and tagged with the keyword “Artificial Intelligenc
13、e” in the Scopus database of academic papers. !9 9x The number of AI papers produced each year has increased by more than 9x since 1996.! ! AI INDEX, NOVEMBER 2017 A comparison of the annual publishing rates of di#erent categories of academic papers, relative to their publishing rates in 1996. The g
14、raph displays the growth of papers across all fields, papers within the Computer Science field, and AI papers within the Computer Science field. The data illustrates that growth in AI publishing is driven by more than a growing interest in the broader field of Computer Science. Concretely, while the
15、 number of papers within the general field of Computer Science has grown by 6x since 1996 the number of AI papers produced each year has increased by more than 9x in that same period. !10! ! AI INDEX, NOVEMBER 2017 Course Enrollment view more information in appendix A2 The number of students enrolle
16、d in introductory Artificial Intelligence & Machine Learning courses at Stanford University. ML is a subfield of AI. We highlight ML courses because of their rapid enrollment growth and because ML techniques are critical to many recent AI achievements. !11 Introductory AI class enrollment at Stanfor
17、d has increased 11x since 1996. Note: The dip in Stanford ML enrollment for the 2016 academic year reflects an administrative quirk that year, not student interest. Details in appendix. 11x! ! AI INDEX, NOVEMBER 2017 We highlight Stanford because our data on other universities is limited. However, w
18、e can project that past enrollment trends at other universities are similar to Stanfords. !12 Note: Many universities have o#ered AI courses since before the 90s. The graphs above represent the years for which we found available data.! ! AI INDEX, NOVEMBER 2017It is worth noting that these graphs re
19、present a specific sliver of the higher education landscape, and the data is not necessarily representative of trends in the broader landscape of academic institutions. !13 Note: Many universities have o#ered ML courses since before the 90s. The graphs above represent the years for which we found av
20、ailable data.! ! AI INDEX, NOVEMBER 2017 Conference Attendance view more information in appendix A3 The number of attendees at a representative sample of AI conferences. The data is split into large conferences (over 1000 attendees) and small conferences (under 1000 attendees) in 2016. !14 These att
21、endance numbers show that research focus has shifted from symbolic reasoning to machine learning and deep learning. Shifting Focus Note: Most of the conferences have existed since the 1980s. The data above represents the years attendance data was recorded.! ! AI INDEX, NOVEMBER 2017!15 Despite shifting focus, there is still a small
