Some questions and concerns in regard to artificial intelligence include:
What is intelligence? How can one recognize its presence and applications? Is it possible for machines to exhibit intelligence?
Does the presence of human-like intelligence imply consciousness and emotions?
Is creating human-like artificial intelligence moral?
The Turing test suggests that a sufficient condition for intelligence is the ability to converse with a human in such a way that the human is fooled into thinking the conversation is with another human.
It seems for example that something could be intelligent even if it couldn't convince anyone of this fact due to language barriers. Others doubt that it is even a sufficient condition. Programs attempting Conversational AI, for example, are learning more and more sophisticated algorithms for sounding intelligent without any actual understanding of the conversations.
A famous though experiment by John Searle argues that AI is impossible. The Chinese room, as he called it argues that syntax is not sufficient for semantics. That symbol manipulation, no matter how complicated, cannot provide genuine meaning or understanding.
The "Chinese Room" is a thought experiment proposed by John Searle in 1980. Assume there is a machine that passes thye Turing Test - there is no way to distinguish this machine from a human being. Now, suppose that the machine passes the Turing Test in Mandarin Chinese. Now, suppose that someone is pen pals with this machine - it writes it letters in Chinese, receives responses in Chinese, and is oblivious to the fact that it is a machine.
Now, assume that you can see this machine's programming. If you followed the exact instructions the machine followed, you would write the same responses to the pen pal's letters as the machine would.
Does this mean that you know Chinese? Assuming you didn't before, of course not - you were blindly following instructions.
The implications of this is that no matter how intelligent a machine appears to be, it isn't. Its instructions give it no more understanding of the world than the Chinese computer's instructions give you understanding of Chinese. This assertion is in direct contradiction to the core assumption of the Turing Test: if it acts human, it is human.
Does this mean Artificial Intelligence is impossible?
There are many ethical issues associated with working to create intelligence. I will not attempt to answer these questions but simply present them for further discussion.
If an AI is comparable in intelligence to humans, then should it have comparable moral status?
Would it be wrong to engineer robots that want to perform tasks unpleasant to humans?
Could a computer simulate an animal or human brain in a way that the simulation should receive the same animal rights or human rights as the actual creature?