Let's start with the basics of how X Window System works. X uses client-server model. An X server program runs on a computer with a graphical display and communicates with various client programs (X clients). The X server acts as a go-between for the user and the client programs, accepting requests for graphical output from the client programs and displaying them to the user (display), and receiving user input (keyboard, mouse) and transmitting it to the client programs.
In X, the server runs on the user's computer, while the clients may run on remote machines. This terminology reverses the common notion of client–server systems, where the client normally runs on the user's local computer and the server runs on the remote computer.