Post by Marti-Pair Furxheir S.H. on Aug 17, 2013 4:44:04 GMT -6
This is actually the second part of last week's lecture and was planned to be presented at the same time, but the first part ended up being longer than planned.
We now know what the 4 main parts of a LAMP server is, and we have a vague idea of how files are processed.
We know, for example, that when a .php file is invoked on the server, Apache finds where the file is, detects it's mine type, and selects the proper handler, in this case, the php compiler.
File Transfers
But how do we get that file on the server in the first place? On your computer, you can use a USB stick, a CD or DVD rom, or use a browser to download a file.
On a server, these options usually remain available, but since in general our web server is not in front of us, we cannot use a mouse and a keyboard to operate a browser and we cannot use a USB stick.
As such, protocols were developed over the years, in succession, to enable us to transfer files to and from servers.
3 of such protocols are in use today: FTP, SCP and SFTP. To understand the last two, we also need to understand that SSH is.
But let's start with FTP, or File Transfer Protocol, which, believe it or not, is a protocol whose specification was published on April 1th 1971. That means that FTP is more than 5 years older than your teacher!
Granted, it's only in June 1980 that FTP began looking like today's FTP, but that's still 15 years before the Internet became somewhat popular.
There is good reason why FTP remain in place for so long: it works. It is efficient, simple, standard and every server on the planet is able to support FTP.
Since the end of the nineties, it is even possible to secure FTP connection because let's face it, the FTP protocol specification mentions that usernames and password are sent in plain text.
FTP is rather limited in its feature set and when SSH came around, it became clear that a more robust and secure File Transfer protocol was needed.
That's where Secure Copy, also called Secure Copy Protocol or SCP, was designed. It works in an encrypted SSH sessions (see below) so that the data transferred is by design secure, unlike FTP which is by design insecure and can be optionally secured (sometimes called FTPS, or FTP over SSL)
SCP is a native Linux command and it's support was never universal.
But it's fast. It's defenders claim that it is by far the fastest of the 3 protocols, and honestly, my experience with all 3 point to that direction.
SCP, while being very different under the hood to FTP, works a lot like its predecessor, and in both cases, they are pretty "dumb" protocols, in the sense that they only do very basic operations.
SFTP (Secure File Transfer Protocol) is also a file transfer protocol built on top of SSH, but it was made primarily to solve the few problems that SCP has, to provide more powerful features and to try and make it platform generic.
Just be careful, there are 2 protocols with the initials SFTP: Secure File Transfer Protocol and Simple File Transfer Protocol but the latter was never widely accepted and is now obsolete.
SFTP, unlike FTP and SCP is more than a file transfer protocol and is more a file management protocol since it includes, built-in, directory listing, file deletion, resuming interrupted downloads, which are all features supported by custom extensions on top of the 2 previous protocols.
Resuming interrupted downloads, for example, was supported by many FTP and SCP programs, but the work was done by these client software and prone to potential problems. SFTP on the other hand built the features directly for more robust operation.
SFTP is also the only file transfer protocol of the list to properly support wildcards. If a directory with 1000 files is requested be deleted, the formers 2 protocols will individually delete each of the 1000 files while SFTP is able to send a single command to delete all.
In short, SFTP is meant to be used alone to manage files whereas SCP is meant to be used alongside SSH.
It also sends meta data with files downloaded, such timestamps and properties, which FTP is not able to do natively.
Even better, SFTP is widely available across all platforms, having a better distribution than SCP.
Still, it wasn't able to fully supplant the predecessor it was means to replace, Why?
1 ) It's not as fast as SCP is.
2 ) Professionals often have an existing workflow in place for SCP where they have access to SSH on top of SCP. WinSCP, for example, supports sending SSH commands from the SCP list of files, solving many of the limitations of SCP.
3 ) From SSH, an SCP transfer is not only faster, it is also easier to use.
But what is SSH??? SSH is the Secure Shell, a network protocol allowing to have a remote command-line terminal.
In short, it allows you to have access to the command line terminal of a remote computer to issue commands. On your Windows computer, it used to be called a DOS prompt, for example.
You can list files, change directories, execute command line programs, compiles scripts, edit files using basic editors like NANO and PICO or more advanced ones like VIM. You can even connect to other servers using FTP, SCP and even SSH.
If you've ever used Telnet to connect to another computer, you have a basic understanding of what SSH does. The different between Telnet and SSH, is roughly the same as the difference between FTP and SCP: SSH is the fully secure alternative to Telnet.
Notably, SSH introduces an alternative to the username and password scheme of FTP and Telnet: public-private keys. Of course, username and passwords are still supported.
The way the new system works, is that a program is used to generate 2 keys: a private one, kept on the server, and a public one, downloaded to the user computer.
When that user later connects to SSH, instead of connecting using a password, it connects using that public key and by sending its own public key.
The remote computer uses the mathematical formula in its own secret private key, combines it with the mathematical formula previously received from the server and the server will decode the information using its own secret private key and the transmitted remote public key.
It is the same basic system used to encrypt SSL connections, but in this case, used as an authentication factor. In an SSL server, the public key is shared publicly by the SSL server, but in an SSH connection, the public key is used in place of password and kept privately.
However, it is still possible to add a password to a Public-Private key pair so that without that password, the public key is useless.
SSH is also widely available across all platforms, but the commands used in SSH vary greatly. To list a directory in Windows, we use the "dir" command, but on Linux, we use the "ls" command. To delete on windows, we use the "del" command, but on Linux, we use the "rm" command.
A lot of common SSH commands can be performed in the WinSCP graphical shell, like changing file permissions.
Rights and Permission
Which brings us to the next chapter of this lecture: Rights and Permission.
On Linux, files permissions are managed using ACL, Access Control List.
On Linux, each file has 3 different type of roles each with its own set of rights: those of the owner, those of the group and those every other users.
Each roles has rights specified for whether the role and read [r], write (modify) [w] or execute [x] the file. When a right is not present, we typically mark it with a -
This means that in the basic ACL system, there are 9 flags combined together, a little like this:
Meanings: The owner is allowed to read, write and execute the file. Members of the group are able to read and execute the file. Everyone else can read the file.
We can also store the value numerically in octal (base 8). We assign Read a value of 4, Write a value of 2 and Execute a value of 1 and add the allowed rights. In the above example, the rights would be 754, often written 0754 because in the C programming language, octal values were prefixed by a 0.
And what are owners and groups? Each file belongs to a specific user and to a specific group. A group can have multiple users.
For example, on a Linux servers, you will have your own files belonging to your own user account, and to probably your own group. Every user you want to share your files with will simply be added to your group. You can be a member of multiple groups, allowing multiple levels of sharing.
For directories, the same rights apply, but to change files in a directory, you need to have write permissions. To list the files of the directory, you need read permissions, and to enter the directory (set it as the current), you need execute permission.
To change the rights on Linux SSH, we use the CHMOD command. To change the owner of a file, we use the CHOWN command and for the group, we use the CHGRP command.
Be warned that on a Linux server, the root user are able to see all the files on the server, regardless of the permissions, just like on a Windows server, the domain controller is usually (but not always) able to do the same.
TCP/IP and ports
That leaves us with only 1 topic for this lecture: TCP/IP and ports.
The Internet uses the TCP/IP protocol and I won't dwell on the subject because there isn't a lot you really need to know to be a good web developer.
You should know however that each TCP/IP connection needs two major parameters: the IP address and the Port of the server.
The IP address is like the postal address of the server you want to connect to. It's a little like the phone number to reach a company.
The Port on the other hand, is like the extension you want to dial or the office you want to visit. They are numbered from 0 to 65635.
The first 1023 ports are "well-known" and fixed: port #80 is usually the web server (with an alternate on port #8080). FTP is usually port 20 (data) and 21 (control) , while SSH/SCP/SFTP are on port 22. Telnet is on post 23.
SMTP is on port 25, DNS is on port 53, POP3 is on port 110, IMAP is port 143, SNMP is on ports 161 and 162, HTTP connections over SSL are on port 443,
But some of the ports above 1023 are also well known: 2082 (HTTP) 2083 (SSL) Cpanel, 2086 (HTTP), 2087 (SSL), WHM, 3306: Mysql Database System.
Some ports are even used by different systems on different platforms.
When you connect to a port, for example, port 22 for SSH, your own computer also opens a random port from which you connect to the other port. You might be connected to port 22 from port 3267.
That's pretty much what you need to know. I know, it's not much, but you don't need to know that much at this point.
And that concludes the 2nd part of this lecture. Next week: Basic HTML.
This week's assignment is in 2 parts:
1 ) You need to connect to a web server, any web server you have legal access to, to upload a file named assignmenta.html which contains on the first line your citizen number (which can be found at www.talossa.ca/files/citizens.php and your Forum name. Mine would be 101 Marti-Pair Furxheir
2 ) On the second line, you need to enter the name of the program you are using (ideally WinSCP but not everyone uses a PC) and the protocol used (FTP, SCP, SFTP).
3 ) On the next lines, one per line, you need to type in ascending order the various ports I have listed, the name of the service typically listening on that port, and a few words description of what it is.
For example, the first 2 entry (freebies) would be:
20 FTP Data Port: The port thru which FTP transfers files. FTP is a basic file transfer protocol used to copy files from 2 computers. This port copies the files.
21 FTP Control Port: The port thru which FTP sends commands. FTP is a basic file transfer protocol used to copy files from 2 computers. This port controls the transfers on port 20.
If I am counting well, that means you have 16 additional ports to document. When a program is on two ports, indicate briefly the difference between the 2.
To send your assignment, do not post it on Wittenberg and instead, send me the link via private message. If you do not have access to a web server, one will be provided for the purpose of this assignment if you contact me by private message. You do not need to wait until your receive your access to start writing the file.
You have until Sunday September 1st to deliver the link, BUT, the assignment WILL be modified next week with additional work to be done (and also due on September 1st)
I strongly recommend that you perform this week's assignment as the assignment next week will assume that this week's assignment is already completed.
We now know what the 4 main parts of a LAMP server is, and we have a vague idea of how files are processed.
We know, for example, that when a .php file is invoked on the server, Apache finds where the file is, detects it's mine type, and selects the proper handler, in this case, the php compiler.
File Transfers
But how do we get that file on the server in the first place? On your computer, you can use a USB stick, a CD or DVD rom, or use a browser to download a file.
On a server, these options usually remain available, but since in general our web server is not in front of us, we cannot use a mouse and a keyboard to operate a browser and we cannot use a USB stick.
As such, protocols were developed over the years, in succession, to enable us to transfer files to and from servers.
3 of such protocols are in use today: FTP, SCP and SFTP. To understand the last two, we also need to understand that SSH is.
But let's start with FTP, or File Transfer Protocol, which, believe it or not, is a protocol whose specification was published on April 1th 1971. That means that FTP is more than 5 years older than your teacher!
Granted, it's only in June 1980 that FTP began looking like today's FTP, but that's still 15 years before the Internet became somewhat popular.
There is good reason why FTP remain in place for so long: it works. It is efficient, simple, standard and every server on the planet is able to support FTP.
Since the end of the nineties, it is even possible to secure FTP connection because let's face it, the FTP protocol specification mentions that usernames and password are sent in plain text.
FTP is rather limited in its feature set and when SSH came around, it became clear that a more robust and secure File Transfer protocol was needed.
That's where Secure Copy, also called Secure Copy Protocol or SCP, was designed. It works in an encrypted SSH sessions (see below) so that the data transferred is by design secure, unlike FTP which is by design insecure and can be optionally secured (sometimes called FTPS, or FTP over SSL)
SCP is a native Linux command and it's support was never universal.
But it's fast. It's defenders claim that it is by far the fastest of the 3 protocols, and honestly, my experience with all 3 point to that direction.
SCP, while being very different under the hood to FTP, works a lot like its predecessor, and in both cases, they are pretty "dumb" protocols, in the sense that they only do very basic operations.
SFTP (Secure File Transfer Protocol) is also a file transfer protocol built on top of SSH, but it was made primarily to solve the few problems that SCP has, to provide more powerful features and to try and make it platform generic.
Just be careful, there are 2 protocols with the initials SFTP: Secure File Transfer Protocol and Simple File Transfer Protocol but the latter was never widely accepted and is now obsolete.
SFTP, unlike FTP and SCP is more than a file transfer protocol and is more a file management protocol since it includes, built-in, directory listing, file deletion, resuming interrupted downloads, which are all features supported by custom extensions on top of the 2 previous protocols.
Resuming interrupted downloads, for example, was supported by many FTP and SCP programs, but the work was done by these client software and prone to potential problems. SFTP on the other hand built the features directly for more robust operation.
SFTP is also the only file transfer protocol of the list to properly support wildcards. If a directory with 1000 files is requested be deleted, the formers 2 protocols will individually delete each of the 1000 files while SFTP is able to send a single command to delete all.
In short, SFTP is meant to be used alone to manage files whereas SCP is meant to be used alongside SSH.
It also sends meta data with files downloaded, such timestamps and properties, which FTP is not able to do natively.
Even better, SFTP is widely available across all platforms, having a better distribution than SCP.
Still, it wasn't able to fully supplant the predecessor it was means to replace, Why?
1 ) It's not as fast as SCP is.
2 ) Professionals often have an existing workflow in place for SCP where they have access to SSH on top of SCP. WinSCP, for example, supports sending SSH commands from the SCP list of files, solving many of the limitations of SCP.
3 ) From SSH, an SCP transfer is not only faster, it is also easier to use.
But what is SSH??? SSH is the Secure Shell, a network protocol allowing to have a remote command-line terminal.
In short, it allows you to have access to the command line terminal of a remote computer to issue commands. On your Windows computer, it used to be called a DOS prompt, for example.
You can list files, change directories, execute command line programs, compiles scripts, edit files using basic editors like NANO and PICO or more advanced ones like VIM. You can even connect to other servers using FTP, SCP and even SSH.
If you've ever used Telnet to connect to another computer, you have a basic understanding of what SSH does. The different between Telnet and SSH, is roughly the same as the difference between FTP and SCP: SSH is the fully secure alternative to Telnet.
Notably, SSH introduces an alternative to the username and password scheme of FTP and Telnet: public-private keys. Of course, username and passwords are still supported.
The way the new system works, is that a program is used to generate 2 keys: a private one, kept on the server, and a public one, downloaded to the user computer.
When that user later connects to SSH, instead of connecting using a password, it connects using that public key and by sending its own public key.
The remote computer uses the mathematical formula in its own secret private key, combines it with the mathematical formula previously received from the server and the server will decode the information using its own secret private key and the transmitted remote public key.
It is the same basic system used to encrypt SSL connections, but in this case, used as an authentication factor. In an SSL server, the public key is shared publicly by the SSL server, but in an SSH connection, the public key is used in place of password and kept privately.
However, it is still possible to add a password to a Public-Private key pair so that without that password, the public key is useless.
SSH is also widely available across all platforms, but the commands used in SSH vary greatly. To list a directory in Windows, we use the "dir" command, but on Linux, we use the "ls" command. To delete on windows, we use the "del" command, but on Linux, we use the "rm" command.
A lot of common SSH commands can be performed in the WinSCP graphical shell, like changing file permissions.
Rights and Permission
Which brings us to the next chapter of this lecture: Rights and Permission.
On Linux, files permissions are managed using ACL, Access Control List.
On Linux, each file has 3 different type of roles each with its own set of rights: those of the owner, those of the group and those every other users.
Each roles has rights specified for whether the role and read [r], write (modify) [w] or execute [x] the file. When a right is not present, we typically mark it with a -
This means that in the basic ACL system, there are 9 flags combined together, a little like this:
rwxr-xr--
Meanings: The owner is allowed to read, write and execute the file. Members of the group are able to read and execute the file. Everyone else can read the file.
We can also store the value numerically in octal (base 8). We assign Read a value of 4, Write a value of 2 and Execute a value of 1 and add the allowed rights. In the above example, the rights would be 754, often written 0754 because in the C programming language, octal values were prefixed by a 0.
And what are owners and groups? Each file belongs to a specific user and to a specific group. A group can have multiple users.
For example, on a Linux servers, you will have your own files belonging to your own user account, and to probably your own group. Every user you want to share your files with will simply be added to your group. You can be a member of multiple groups, allowing multiple levels of sharing.
For directories, the same rights apply, but to change files in a directory, you need to have write permissions. To list the files of the directory, you need read permissions, and to enter the directory (set it as the current), you need execute permission.
To change the rights on Linux SSH, we use the CHMOD command. To change the owner of a file, we use the CHOWN command and for the group, we use the CHGRP command.
Be warned that on a Linux server, the root user are able to see all the files on the server, regardless of the permissions, just like on a Windows server, the domain controller is usually (but not always) able to do the same.
TCP/IP and ports
That leaves us with only 1 topic for this lecture: TCP/IP and ports.
The Internet uses the TCP/IP protocol and I won't dwell on the subject because there isn't a lot you really need to know to be a good web developer.
You should know however that each TCP/IP connection needs two major parameters: the IP address and the Port of the server.
The IP address is like the postal address of the server you want to connect to. It's a little like the phone number to reach a company.
The Port on the other hand, is like the extension you want to dial or the office you want to visit. They are numbered from 0 to 65635.
The first 1023 ports are "well-known" and fixed: port #80 is usually the web server (with an alternate on port #8080). FTP is usually port 20 (data) and 21 (control) , while SSH/SCP/SFTP are on port 22. Telnet is on post 23.
SMTP is on port 25, DNS is on port 53, POP3 is on port 110, IMAP is port 143, SNMP is on ports 161 and 162, HTTP connections over SSL are on port 443,
But some of the ports above 1023 are also well known: 2082 (HTTP) 2083 (SSL) Cpanel, 2086 (HTTP), 2087 (SSL), WHM, 3306: Mysql Database System.
Some ports are even used by different systems on different platforms.
When you connect to a port, for example, port 22 for SSH, your own computer also opens a random port from which you connect to the other port. You might be connected to port 22 from port 3267.
That's pretty much what you need to know. I know, it's not much, but you don't need to know that much at this point.
And that concludes the 2nd part of this lecture. Next week: Basic HTML.
This week's assignment is in 2 parts:
1 ) You need to connect to a web server, any web server you have legal access to, to upload a file named assignmenta.html which contains on the first line your citizen number (which can be found at www.talossa.ca/files/citizens.php and your Forum name. Mine would be 101 Marti-Pair Furxheir
2 ) On the second line, you need to enter the name of the program you are using (ideally WinSCP but not everyone uses a PC) and the protocol used (FTP, SCP, SFTP).
3 ) On the next lines, one per line, you need to type in ascending order the various ports I have listed, the name of the service typically listening on that port, and a few words description of what it is.
For example, the first 2 entry (freebies) would be:
20 FTP Data Port: The port thru which FTP transfers files. FTP is a basic file transfer protocol used to copy files from 2 computers. This port copies the files.
21 FTP Control Port: The port thru which FTP sends commands. FTP is a basic file transfer protocol used to copy files from 2 computers. This port controls the transfers on port 20.
If I am counting well, that means you have 16 additional ports to document. When a program is on two ports, indicate briefly the difference between the 2.
To send your assignment, do not post it on Wittenberg and instead, send me the link via private message. If you do not have access to a web server, one will be provided for the purpose of this assignment if you contact me by private message. You do not need to wait until your receive your access to start writing the file.
You have until Sunday September 1st to deliver the link, BUT, the assignment WILL be modified next week with additional work to be done (and also due on September 1st)
I strongly recommend that you perform this week's assignment as the assignment next week will assume that this week's assignment is already completed.