1. Questions

ChatGPT, a generative artificial intelligence (AI) chatting machine developed by OpenAI (OpenAI 2023a), has been explosively stimulating worldwide discussion about its capabilities and limitations since its launch in November 2022 (Thorp 2023). A prominent discussion centers around ChatGPT’s impacts on higher education and science (van Dis et al. 2023). To initiate such conversation within transport practice and research communities, we rapidly undertook an exploratory study motivated by a simple question: How does ChatGPT introduce transport problems and solutions in North America?

2. Methods

We asked four questions (prompts) about identifying and tackling transport challenges in the United States and Canada to ChatGPT:

  • Q1: What are the important transportation issues of the United States?

  • Q2: What are the solutions to transportation issues in the United States?

  • Q3: What are the important transportation issues of Canada?

  • Q4: What are the solutions to transportation issues in Canada?

Figure 1 shows one of the answers we obtained from ChatGPT. We utilized the most updated version of ChatGPT (January 30, 2023) at the time of writing, using a default setting for the analysis. We independently repeated each inquiry five times using the “Regenerate Response” feature to obtain multiple answers to see the reliability of ChatGPT.

Figure 1
Figure 1.A screenshot of a sample response from ChatGPT

Using the answers for each question as a basis, we created a word cloud to summarize the ChatGPT’s responses using R after removing trivial stop words. We assumed that words appearing more frequently are relatively more important than other words in ChatGPT’s answers (Green et al. 2020; Kim, Wang, and Rapuri 2022). More frequent words are visualized in a bigger size in word clouds. Additionally, we qualitatively assessed the ChatGPT answers to see if they aligned with our general expectations based on the domain knowledge.

3. Findings

Table 1 presents the word counts for the ChatGPT’s answers, demonstrating that ChatGPT gives responses having a similar number of words for each question. Moreover, by qualitatively observing responses obtained from five attempts, we conclude that responses (in terms of their contents and topics) are reasonably similar.

For both the United States and Canada, the average word counts are greater in the solution prompts (Q2 and Q4) than in the issue prompts (Q1 and Q3). Moreover, ChatGPT provides a longer answer to questions related to the United States than those related to Canada. One possible explanation can be the potential geographic biases in the quantity and quality of the ChatGPT training data (Graham et al. 2014; Mandal, Leavy, and Little 2021) between the United States and Canada. However, future studies are needed to investigate this issue more deeply.

Table 1.Word counts of the ChatGPT’s answer in terms of questions and requests.
Requests
1st 2nd 3rd 4th 5th Average
Questions Q1 171* 170 187 176 181 177.0
Q2 219 218 220 224 220 220.2
Q3 143 134 128 169 131 141.0
Q4 168 179 191 140 189 173.4

* Note: This indicates there are 171 words in ChatGPT’s answer to Q1 in the first request.

Figure 2 shows ChatGPT’s answers to Q1 and Q2 at one attempt. Figure 3 illustrates the word clouds of the ChatGPT’s responses. ChatGPT identified eight major transport issues in the US, including 1) congestion, 2) aging infrastructure, 3) climate change, 4) funding shortage, 5) the need to balance the interests of different transport modes, 6) the need to enhance intermodal connectivity, 7) the growing demand for alternative modes of transport, and 8) the need to reduce greenhouse gas emissions. Overall, the transport issues suggested by ChatGPT are in line with what transport researchers have discussed (NASEM, 2018).

Figure 2
Figure 2.ChatGPT’s answers to Q1 and Q2 (Selected).
Figure 3
Figure 3.Word cloud results for ChatGPT’s answers to (a) Q1, (b) Q2, (c) Q3, and (d) Q4.

ChatGPT also suggested ten solutions to solve the transport issues in the United States. These solutions include 1) investment in public transit, 2) implementation of smart transport systems, 3) development of alternative modes of transport, 4) improving infrastructure, 5) collaboration between different levels of government, the private sectors, and the public, 6) incorporating sustainability and resiliency, 7) encouraging private investment, 8) enhancing intercity and international trade, 9) investment in low-carbon transport options, and 10) increasing access to affordable transport options.

It is interesting to note that ChatGPT’s solutions address sustainability, resiliency, affordability, and multi-modality, which is in line with a recent empirical study that revealed ChatGPT’s pro-environmental and left-libertarian orientations (Hartmann, Schwenzow, and Witte 2023). Transport researchers might be relieved that ChatGPT did not suggest “awful” solutions, such as highway expansions or cutting public transit services. Overall, these solutions provided by ChatGPT generally echo transport researchers’ understanding of transport systems in North America.

We do not observe substantial differences when comparing ChatGPT’s responses to Canada with those to the United States. However, we note that ChatGPT might not always generate an intuitive answer. For instance, the frequency of the word “congestion” is higher in ChatGPT’s answers to Canada’s transport issues (Figure 3[c]) when compared to its frequency in ChatGPT’s answers on the United States (Figure 3[a]) despite the fact that traffic congestion is typically more severe in United States cities than in Canadian cities (INRIX 2022). However, we acknowledge that the higher frequency of a specific word does not necessarily mean its higher importance as a transport issue. Thus, advanced text mining techniques (e.g., semantic analysis) can be leveraged to draw a more definite and robust conclusion.

How does ChatGPT introduce transport problems and solutions in North America? Our results revealed that transport issues and solutions introduced by ChatGPT generally align well with transport researchers’ expectations. We argue that ChatGPT might be a decent starting point for citizens and researchers to discuss transport issues and solutions. Specifically, thanks to the conversational format of ChatGPT, citizens could explore information in a more interactive way (e.g., asking follow-up questions) compared to when they use traditional web-based search engines. However, one should be aware that ChatGPT’s capability may be limited in providing trustworthy or sound solutions because of the potential issues (e.g., geographic biases, inaccuracy) in its training data.

As an early effort, our research contributes to the transport literature by conducting a rapid assessment of ChatGPT’s capabilities for introducing and summarizing transport issues and solutions in North America. Transport researchers are encouraged to expand our work by considering potential analytical pitfalls, such as the black box characteristics of ChatGPT, the impacts of prompt wording on its responses, and performance differences between a free and premium model that has been recently released (OpenAI 2023b). A more comprehensive fact-checking of ChatGPT’s responses based on academic articles or government reports can be another viable future research direction. As generative AI technologies will continue to evolve with great potential to revolutionize our society, we do call for transport researchers and practitioners to think collectively about whether and how to embrace this game changer.

Acknowledgement

Junghwan Kim is supported by the Institute for Society, Culture and Environment at Virginia Tech. Jinhyung Lee is supported by the Faculty of Social Science at Western University.