The flexibility and simple operation allows Netcat to fill a niche when it comes to moving a file or files in a quick and easy fashion. Encryption is provided via several different avenues including integrated support on some of the more modern Netcat variants, tunneling via third-party tools, or operating system integrated IPsec policies. After the file is transferred, any of several hash-generating programs can be used to verify the integrity of the file you received. Instead of being a specialized tool, like FTP, Netcat is a general-purpose tool, much like the Swiss army knife it is so often compared to. As a general purpose tool, the uses for Netcat are often more limited by your imagination than by Netcat’s functionality.
FT Using Netcat (the long story)
Netcat's Cons
An area of concern is that of user and rights management. If you want to strictly control user access and user rights to the transferred files and the upload directories, Netcat is probably a poor choice.
Netcat itself offers no way to control who can send what to whom.
- In order to exercise tight control over file uploads you would be forced to configure your permissions using the file system itself, which could be administratively burdensome in a complex environment.
- Additionally, such configurations also increase the likelihood of introducing human error.
- If i.e. you decide to grant the rights to maintain these permissions to another person, doing so may entail granting them more access to the underlying file system than you had in mind.
A product designed to give you granular control over user access rights such as a good FTP server, would make this task much easier, and probably be a better choice.Another area where security can become a concern is the fact that some programs are more easily used by an attacker to compromise security on the host system.
- In other words, it is true that an FTP server can be used against you by an attacker, however, the configuration of the FTP server is probably much more closely guarded.
- The configuration files are probably well secured and logging might notice attempts to manipulate the software’s normal configuration.
Netcat is a tool hackers are extremely familiar with. Simply having it on a host where it could possibly be leveraged by an attacker is an increased risk.High security hosts will often have policies expressly prohibiting such programs from even being on the system. Of course Netcat can be secured appropriately,
it simply represents an increased risk over some of the less easily abused file transfer services.
Complex environments issues
If you will be using Windows hosts as either the client or server for your file transfers, this means you will have to “install” the appropriate Netcat version. Understandably, the “installation” may be as simple as copying a file onto the system, but even this process could be slowed by bureaucracy and cause problems on some mission-critical servers.
Sometimes simply getting permission to install non-Microsoft software can be challenging. Considering this, in some environments the simple fact that FTP clients are installed on all modern Microsoft operating systems, as well as being supported in Internet Explorer browsers, Netcat might not be the best option.
Even after the installation is complete, there is the matter of configuring the transfer. You may be wondering what there is to configure; you enter a single line at each end and off you go. It might be that simple, especially for a one-off transfer, but if you want something to be run regularly and programmatically it will probably take more time than this.
You might need to configure some type of wrapper script to handle errors or problems, as well as something to verify the file integrity and permissions.
- How do you handle a corrupted file?
- How do you deal with an interrupted download?
If you can get Netcat installed where you need it without issues, there are still roadblocks. If all you need is a simple no-frills file transfer, configuring any needed scripts should be pretty trivial. If you need to configure some more complex routing to handle and rotate logs, interact with syslog, or deal with directories and rights, it could take some time.
The point here is that even if the bureaucratic process doesn’t slow you down, the developmental one might. Testing and troubleshooting file transfer scripts can easily end up taking more time than you have. All of these things collectively mean you might end up spending more time troubleshooting the simple Netcat transfer, when an alternate solution could have been implemented more quickly.
Netcat's Pros
There are still many circumstances where Netcat is not only a viable choice for moving data, but a preferable one.Speedy Gonzales
Let’s suppose you need to move a file over from one host to another and it needs to be done quickly. Setting up an FTP or Trivial File Transfer Protocol (TFTP) server and creating users and setting up access rights, while not difficult, seems like a lot of work to just move one file.
Perhaps one host is a Linux host and one is a Windows host. To make matter worse, let’s say that Windows file sharing is not permitted on the network and the file you need to send it is too large for the e-mail system. Sounds like a real hassle; however, this scenario is fairly common in many corporate settings. You might find yourself in a position where preparing for the file transfer is going to take far longer than the actual transfer itself.
Here's when Netcat comes to the rescue. With the cross-platform support, you can run the client or server from your pen drive and have your file moved in less time than it takes to even locate the form required for setting up the appropriate service.
Netcat’s simplicity means those one-off file transfers can be accomplished very rapidly and with the minimal amount of fuss.
Sometimes you may not have the privileges required to create an FTP server or enable file sharing, but more often than not you will be able to start up Netcat on some high-numbered port. Even a user with minimal privileges can successfully move a file with Netcat. This single feature, rapid deployment, is reason enough to keep Netcat in your networking toolbox.
Stealth
It could be a matter of policy, a penetration testing exercise, or other grey hat endeavors, however, the ability to stealthily send data across the port of your choosing can be invaluable. Netcat not only grants you the ability to send data, but also allows you to push the data in either direction, irrespective of the direction the session was initiated from.
While many of the more sophisticated file transfer protocols are limited to a standard set of ports, Netcat has no such limitations. If the only port open is User Datagram Protocol (UDP)/Domain Name Service (DNS) port 53, you can certainly connect outbound over port 53 and then pull data back into your client system.
Even if the port you choose is being closely monitored for content, FTP and other such protocols contain easily identifiable characteristics that your Netcat file transfer may lack, potentially making a Netcat session harder to detect. This becomes even more true when you introduce simple and painless encryption to your Netcat session. (See the Cryptcat, OpenSSH, OpenSSL, IPsec) Of course there are some solutions that can hide the data even better, such as a Hypertext Transfer Protocol Secure (HTTPS) proxy, but typically these take a little more time and effort to configure and get working.
If you combine a client script with “port knocking,” you have the recipe for some seriously stealthy data transfers. See portknocking.org if you are not familiar with the powerful security benefits of port knocking
Small Footprint
Here're environments where production servers could not have software installed or removed except during pre-defined maintenance windows. Some company policies, however, will allow for simple portable applications that require no registry changes or file system access changes to be run on an as-needed basis. Some change management policies will allow the use of Netcat while other will not. You will need to determine what is appropriate in your environment. Even if you do not have the luxury of being able to start up Netcat without prior approval, Netcat offers a very small footprint, with the original version weighing in around 60KB. This, and the open source nature of many Netcat variants, may allow you to implement Netcat when some other “heavier” solution would have been problematic. Given Netcat’s small size and minimal system impact, it can find a special niche when such traits are important.
Simple Operation
The relatively simple operation of Netcat means it can be run with very little training. Setting up an FTP server on the other hand, takes a little more know how, especially if firewalls are involved. Explaining to the person at the other end how to specify passive FTP in place of active FTP can be challenging and frustrating, even if you are using a GUI.
If you manage to overcome that hurdle, you could still find yourself troubleshooting user rights. Do you have access to the local file system? Is granting access feasible or permitted? Netcat’s options are simple enough. An e-mail with the attached executable would probably be all it took to get a file transferred with no more than a neophyte at the other end for help. The nearly identical operation between operating systems also means that familiarity with Linux Netcat pretty much means familiarity with Windows Netcat.
Overall, Netcat fills a special role when it comes to file transfers. When you need a way to move a file and need it done yesterday, Netcat may be the single fastest solution available. The lack of security features makes Netcat unsuitable for some tasks, but also ideal for others. You must evaluate the pros and cons of your particular circumstances. The right balance must be found between features and ease of use.
Your comfort level with implementing a given solution is also a consideration. If you know Secure file CoPy (SCP) inside and out, maybe that is easier than using Netcat. Netcat’s simplicity and ease of use could mean the difference between getting the job done “on time,” and getting it done “too late.”
Note:
Before setting out to use a particular flavor of Netcat, consider your intended application. Not all of the Netcat varieties will run on both Windows and *nix systems.
- The original Netcat, GNU Netcat, cryptcat, and SBD all have both Windows and UNIX/Linux versions.
- The Socat offer *nix versions only, with no Windows support as of this writing.
Performing Basic File Transfers
Not all environments require high security, and not all file transfers will contain confidential or otherwise sensitive information. In some cases, your needs will be simple. To accommodate this we will start by looking at file transfers using Netcat in it’s most basic format. While not very robust, it serves its purpose and gets the job done.
Transferring Files with the Hobbie's Netcat
On the server side you will need to: - specify that Netcat should be listening for inbound connections with the –l option, and
- specify which port to listen on via the –p option.
- You also need to specify the file to direct the output to.
$ nc -l -p 4444 > /test/outfile.txt
This command would tell Netcat to listen on port 4444 (over TCP by default), and send the output from the connection to /test/infile.txt.
On the client side, the format is similar. All you need is:
- a destination IP address and port number, as well as
- the file you wish to redirect to the network socket.
$ nc 192.168.1.99 4444 < C:\test\infile.txt
or
$ cat /test/infile.txt | nc 192.168.1.99 4444
On *nix systems you can use the cat command to read the file into Netcat or the </> redirection. Here we will use the </> format because they will work on Windows or Linux.
This would connect to the host at 192.168.1.99 over port 4444 and send the C:\test\outfile.txt over the socket thus created. That is all it takes to move a file.
- The file in question does not have to have the same name at each end, though it can.
- The direction of the file transfer is from the client to the server in this example and all examples used here.
- You can also “push” a file from the server to the client just as easily, by reversing the direction of the </> symbols.
The –p option specifying the port will work with or without a space between the –p and the port number.
You can use the –u option to use Netcat over UDP instead of TCP, though for file transfers this is rarely needed or advisable.
Warning:
When using the redirect option to transfer a file, you will receive no warning or error if you attempt to overwrite an existing file.
You can use a >> to output to a file instead of > , and the file will be appended to instead of overwritten.Beyond these simple steps, it is left to the user to ensure that old data is not inadvertently overwritten via Netcat’s redirection capabilities. If this is a concern, see some of the Netcat variants that offer more control over how files are handled.
Closing Netcat When the Transfer is Completed
One of the first things you will notice as you play with Netcat, is that the file never seems to finish. At least it gives no indication that the file transfer is complete. Unfortunately, it will sit there indefinitely. If you break out (normally CTRL+C), you will find that the file is there in it’s entirety, assuming you allowed enough time to complete the transfer. This raises the question of how do I make the socket close when the file transfer is complete?If you use the –w option, it will allow you to specify the delay in seconds after the end of the file is reached before Netcat will close the connection.
On Windows systems, the –w option can be used on the server or the client with identical results.Both of these commands would result in the connection closing five seconds after the file transfer is completed. Here are commands for:
Windows server side:
$ nc –l –w5 –p 4444 > /test/infile.txt
and Windows client side:
$ nc –w5 192.168.1.99 4444 < C:\test\outfile.txt
The behavior of the –w option is a little different on Linux. On Linux, the –w option used on the Netcat server specifies how long to wait for a connection before closing. This can be useful if you want to only keep the port open for a short time before closing it, perhaps to increase security. The –w option used on the client side will act as a timeout before the connection closes, but after the file transfer, the same as on Windows systems.This means the –w option behaves differently depending on which end of the connection you use it on (as the help output describes). This will cause the listening server to wait five seconds for a connection to be made before shutting down the listening port. If –w is used only on the server, the connection will not close by itself and will instead stay open when the file transfer is completed.
$ nc –l –w5 –p4444 > /test/infile.txt
This will cause the client to send the file specified, and then wait five seconds after the file is transferred before shutting itself down.
$ nc –w5 192.168.1.99 4444 < C:\test\outfile.txt
When you are mixing Linux and Windows Netcat servers and clients, you get the above behavior based on who is the client and who is the server.
In other words, because the –w option as an initial connection timeout is a server-based option, it will act as a connection timeout if Linux is the server. If it is a windows server, it will act as an end-of-file timeout.Either system will honor the end of file (EOF) timeout as a client option.
Other Options and Considerations
The –L option is specific to the Windows version of Netcat when used in server mode. This tells Netcat to continue listening on the indicated port even after the client disconnects. This is a departure from the standard behavior, which is to shut down the server side when the client disconnects. This option could be useful:- if you want to send data to the server from multiple clients, or
- if you want to send multiple files to the server over different sessions to be appended to the same output file.
are using for redirection will be locked by the file system. This means an attempted move or renaming of the redirected file while Netcat is still running will result in an error.
Linux does not share this behavior, and will happily let you rename the file out from under Netcat. The file locking behavior may sometimes work to your advantage. Suppose you are sending a Windows host some type of log, perhaps from a custom written script. You can have an automated batch file loop and attempt to rename the file with the current date, for example, and you will not have to worry about figuring out if the Netcat portion of the script is completed or not. If Netcat has not closed, you will not be able to rename the file.
The –q option is only available on Linux and has no effect when run on the server, at least as far as file transfers are concerned. When used on a Linux client for a file transfer, it has the effect of closing the connection as soon as the file transfer is completed regardless of the number you specify, much the same as using –w1 on the client would (using zero seconds for the wait time will result in an error). This behavior is a little odd, but considering the option it is intended to be used with, stdin, it really isn’t that surprising.
Timing Transfers, Throughput, etc...
An additional option that could be of benefit when scripting file transfers, is the –v (for verbose) option. This tells Netcat to give you more feedback on the connection process. With a single –v, Netcat will tell you the IP and port being used. With an additional v, Netcat will also tell you the amount of data transferred in bits, similar to the following:
> F:\netcat>nc -v -l -p4444 > F:\small.txt listening on [any] 4444 ... connect to [192.168.1.99] from (UNKNOWN) [192.168.1.112] 53442: NO_DATA
> F:\netcat>nc -vv -l -p4444 > F:\small.txt
listening on [any] 4444
...
connect to [192.168.1.99] from (UNKNOWN) [192.168.1.112] 53492: NO_DATA
sent 0, rcvd 147Note the final line that is added with the second v. This can be particularly handy if you want to grab the file size after the transfer. If necessary, you could script the timestamp before and after the transfer, and combined with the file size and a little math you could determine the throughput, much like an FTP file transfer.
The Windows 2000 resource kit (and likely others) includes timethis.exe, which will accept another command as an argument and time how long it takes to complete the command. As an example, timethis.exe produces the following output for a netcat file transfer:
> timethis “nc 192.168.1.99 4444 < C:\test.txt” timeThis : Command Line : nc 192.168.1.112 4444 < F:\test.txt timeThis : timeThis : Start Time : Tue Apr 01 17:50:21 2008 End Time : Tue Apr 01 17:50:24 2008 timeThis : Elapsed Time : 00:00:02.021
Remember that you will need to time the transfer on the client host.If you are planning on the server continuing to listen after the file is transferred, timethis would never see the command “complete” and know to stop the timer. Even if you are allowing the server to close after the transfer is completed, there would still be the extra time recorded while the server was listening, but before the client connected. This additional time would skew your total time and alter any calculations for throughput you might do.
Linux also offers the(pipe viewer) pv utility. If you do not already have it installed, it has some very cool options for monitoring the performance of a pipe. You can use the
- –b option to display the amount of bytes transferred, and
- the –t option to show the elapsed time.
$ cat test.txt | pv –bt | nc 192.168.1.99 4444 177MB 0:00:19
Tunneling a Transfer Through an Intermediary
Another option is to use a host in between your client and server to bounce the Netcat session off. This could be helpful if you need to change the port you are using somewhere in the middle of the data path. While not supported natively, you can accomplish this by piping Netcat into Netcat on the intermediary system as follows:
$ nc -l -p4444 | nc 192.168.1.99 6666
This uses the same syntax you are familiar with, but sends the output of the listening Netcat into the client session as input. The creative uses you can put this type of simple redirection too are nearly endless. Netcat has what it takes to move a file easily from one system to another without too much fuss. While not sophisticated in it’s capabilities, it can still get the job done.
Of course, these simple file transfers are only the tip of the iceberg. You may still need to find ways to handle file integrity, because Netcat does not verify your data was transferred without errors. You will also want to ensure that the data you are transferring is protected from authorized viewing via encryption. Both of these options will be discussed later here. The next step is to examine some of
the Netcat variants and what options they offer when it comes to transferring files.
Using Netcat Variants
The original Netcat was created over a decade and half ago. As fast as technology moves, that makes it practically ancient by modern standards, and it is a credit to its creator that Netcat is still in wide use today. Despite this, there have been many other talented people who wanted some feature or function that Netcat could not offer. In other instances, popular *nix distributions have continued to develop their own version of the original Netcat, with new features and functionality. For these reasons many alternate versions of Netcat have been developed.
Some are extremely close to the original, offering only a single new feature such as cryptcat, while others introduce a host of options such as Socat. We will explore the file transfer capabilities these variants provide and discuss any caveats to performing file transfers that may exist.
Cryptcat
Cryptcat is a Netcat variant that is very close to the original; in fact the help screens are nearly identical.The only differences from an options perspective are that cryptcat does not offer the –t option or the –q option.
- The –t option tells Netcat to use Telnet negotiation, making Netcat a Telnet client, and
- the –q option is used as an stdin timeout.
The –k option is used to specify a shared secret password. Cryptcat with then use this password as the key for encrypting the data stream using B. Schneier's twofish encryption.
Cryptcat is available for *nix and Windows. Unfortunately they do not offer a pre-compiled executable on the cryptcat home page. If you don’t mind downloading a pre-compiled version from another source, there are many available on the Internet such as the one from www.security
forest.com/downloads/cryptcat.exe.
The operation of cryptcat is simple, following the same syntax and format as Netcat. Traffic through cryptcat will be encrypted even if you do not specify the –k option, though in this case it will use a default key of “metallica.” Otherwise, you can specify the same key on the client and server as follows:
Server:
$ cryptcat –l –k secretkey –p4444
client:
$ cryptcat –k secretkey 192.168.1.99 4444
The following output from tcpdump demonstrates why encryption might be a good idea. The first is the hex output of the packet from Netcat (minus the header data), while the second is the hex output from the same test data “this is a test string,” using the default key with cryptcat.
0x0000: 4500 004a 9806 4000 4006 1e77 c0a8 0170 E..J..@@..w...p 0x0010: c0a8 0170 adc0 115c 355d bdd7 3503 712f ...p...5]..5.q/ 0x0020: 8018 0101 846d 0000 0101 080a 00e2 2a35 ...p...5]..5.q/ 0x0030: 00e2 25d2 7468 6973 2069 7320 6120 7465 ..%.this.is.a.te 0x0040: 7374 2073 7472 696e 670a st.string.
0x0000: 4500 0044 bf71 4000 4006 f711 c0a8 0170 E..D.q@.......p 0x0010: c0a8 0170 adc1 115c 3625 8ca7 368c b1d5 ...p...6%..6... 0x0020: 8018 0101 8467 0000 0101 080a 00e2 37d6 ....g........7. 0x0030: 00e2 37d6 9b4c 4768 3576 71ed f564 bea4 ..7..LGh5vq..d.. 0x0040: 6760 8e1d
Just like Telnet, anyone who was able to capture your Netcat session would be able to see everything that was sent over the link. Cryptcat also supports the –L option on the Windows version. On Windows, the cryptcat executable can even be renamed to nc.exe if you wanted to “upgrade” current Netcat scripts to include encryption. If you need a quick way to move files while still protecting the data from prying eyes, cryptcat might fit the bill nicely.
Default Encryption Keys
As is always the case with any encryption mechanism, avoid using the default encryption keys for anything other than testing functionality. Much the same as the default passwords to common accounts are well known and exploited, so too are default encryption keys. If an attacker stumbled upon your encrypted Netcat stream, he/she could capture it and decrypt it using the default key of “metallica.” This attack does require some skill and opportunity, which may not always be practical, but it simply isn’t worth the risk. Always specify a high quality shared secret key for cryptcat. Another tip is to remember that if you are going to script a repeating process, be sure and take into account the fact that you will need to change the encryption key regularly.
GNU Netcat
The GNU Netcat is a completely rewritten version of the original Netcat licensed under the GNU general public license. Currently GNU Netcat is available for *nix operating systems only, though hopefully it is only a matter of time before someone creates a Windows port.One of the objectives of the project is compatibility with the original Netcat. In addition to this there are some new features that have been added. Overall, the changes from the original Netcat are minimal.
- The –q (stdin timeout) option has been changed to the –c option, but otherwise behaves the same.
- The –L option used in the Windows Netcat now serves a different purpose for the GNU Netcat. While there are a few more changes to the GNU Netcat options, –L option is the only new option of note when it comes to file transfers. The –L option is used to tunnel a connection using Netcat.
$ nc –L 192.168.1.99:6666 –p4444
$ nc -l -p4444 | nc 192.168.1.99 6666
The first of the two examples uses the GNU Netcat tunneling feature. The –p
option specifies the listening port as usual, and the –L option specifies the host and port to forward to.
The second example accomplishes the same thing by piping the input of one instance of Netcat into the output of another instance. This second example will
work with virtually any version of Netcat. This capability could be advantageous when you need to transfer a file, but there is more than one set of firewalls between the source and destination. The tunneling feature could allow you to change the port you use in the middle, increasing the odds you could make the connection successfully.
SBD
Shadowinteger’s Backdoor (SBD) is another Netcat variant, with all of the features of the original Netcat plus some new ones. SBD is available for both Windows and *nix, with pre-compiled binaries included in the single g-zipped file from http://security.cycom.se/dl/sbd. This feature alone makes SBD a good candidate to settle on as your Netcat-like utility of choice. If that is not enough reason, it offers encryption like cryptcat, and a respawn option, providing the –L functionality for both Windows and *nix hosts.You can put a wrapper script around any version of Netcat to cause it to restart on *nix, but with SBD providing that functionality built-in, there is little need for such effort. While the basic operation of SBD remains much the same as the other Netcat-like tools, there are a couple of additional options that could be useful for transferring files.
- SBD allows you to specify the source port via the –p option if you are running in client mode (in server mode, –p specifies the listening port as usual). This feature could allow you to get through a firewall by making the SBD session look like a reply to a friendly protocol such as DNS. Of course, this trick is not going to fool a statefull firewall, but having this increased control never hurts.
- The –r option allows you to tell SBD to re-spawn after a client disconnect instead of the default Netcat behavior, which is to shut down the server.You can also configure a delay in seconds before re-spawning, this delay could be useful to help slow down any attempts to brute force your listening process. The Windows version of Netcat provides the same functionality via the –L option, while the original *nix version offers no such option. The delay can be set to zero seconds which provides identical behavior to the Windows Netcat –L option.
- With cryptcat, encryption is enabled by default, even if you do not specify a shared secret key. SBD takes the same approach except encryption can be disabled completely via the –c option (–c off disables encryption, –c on is the default).
- Similar to cryptcat, the –k option is used to specify the shared secret key.
- Another interesting option is –P, which is used to specify a prefix for incoming data. The original intent seems to be to facilitate SBD as a primitive chat client; however, it could also be useful to provide a sort of data “tag” for the server to log. Take for example, a scenario where you have two or more systems running a custom script you created. You want each system to log a status message so you know the process completed successfully. You could program in a custom message on each host, or grab a variable from the environment, or you could also have SBD add the “tag.”
- The only other consideration to be aware of concerns SBD’s behavior after a file transfer is completed. Unlike the original Netcat, SBD closes the connection after the file transfer is completed. This behavior makes the use of the –w option obsolete.
Socat
Socat (short for socket cat) is by far the most advanced Netcat variant covered.
Socat comes with an extensive number of options (try “Socat -???” for a list). There is currently also a Windows version available, but the impressive list of features could bring socat to the forefront when you are selecting a Netcat-like tool to use. The number of options is far too great for us to explore them all, so we will limit the discussion to those options that are most useful for transferring files. You can download socat from www.dest-unreach.org/Socat/download/. There is also a manual(in html). The examples at the bottom of the page are particularly valuable, though they could stand a little elaboration.
Socat Basics
While basic connectivity with Netcat has been covered extensively by this point, Socat is different enough to justify starting with the absolute basics. To open a simple TCP client connection, you would use the following command:
$ socat tcp:192.168.1.99:4444 stdin
At the server side you would use the following;
$ socat tcp-listen:4444 stdin
The above commands would create a session that would behave the same as a standard Netcat session. As you can see, Socat uses a format of Socat at all times. Basically, Socat acts as the bridge to link the two data channels together. Netcat behaves much the same way, except that Netcat assumes stdin as one of the communication channels. Socat makes no such assumptions, and if you fail to explicitly configure both communication channels, Socat will generate an error.
Transferring Files with Socat
The following commands can be used for a basic file transfer.
On the client:
$ socat tcp:192.168.1.99:4444 open:/test.file
On the server:
$ socat tcp-listen:4444 gopen:/newtest.file
These commands will behave similarly to Netcat. For example, using the above commands, the socat instance will close when the file transfer is complete. The open and gopen options have some subtle differences, however.
Using gopen (generic open) if the file does not exist it will be created. If the file does exist, the new data transferred would be appended to the existing file. Using open on the other hand, will generate an error if the file doesn’t exist. You can avoid this and cause open to create a file that doesn’t exist if you also use the creat option. The default behavior of open is to overwrite a pre-existing file unless you use the append option.
The benefit of using the open option instead of gopen is that you can toggle the append behavior off or on as you like. If you put all of this together it means that for the server instance, both of these commands will operate the same.
$ socat tcp-listen:4444 gopen:/newtest.file
$ socat tcp-listen:4444 open:/newtest.file,creat,append
If you want to ensure that data cannot accidentally be sent back over a given connection in the wrong direction, socat also lets you control that as well. You can invoke Socat in unidirectional mode to ensure that a given data channel is used for reading and the other for writing. The following command would ensure that test.file would be sent from the client, and opened read-only for added security.
On the client:
$ socat –U tcp:192.168.1.99:4444 open:/test.file
On the server:
$ socat –u tcp-listen:4444 gopen:/newtest.file
On the server, the newtest.file would be opened write only and would not be read
from. The –U option specifies that the first channel is for writing, while the second channel is for reading data. The –u option does the same thing, but reverses which channel is for read and which one is for writing.
Socat also offers extensive logging capabilities and is the only Netcat variant which supports syslog natively. The –ly option will cause Socat to generate system messages with a default facility of “ daemon,” though you can specify the facility after the –ly option (–ly[
With all of these powerful features you might be wondering where and how socat
can provide encryption. Socat supports encryption of the data stream but it does not do so natively.
Socat does not have encryption capabilities built into it like cryptcat and SBD. Socat uses third-party tools, in this case OpenSSL, to provide encryption functionality. If you do not have OpenSSL installed on your host, the encryption options of socat will not work.
On the client:
$ socat openssl:192.168.1.99:4444,cert=file.pem,cafile=file.crt stdin
$ socat openssl-listen:4444,cert=file.pem,cafile=file.crt stdin
$ socat TCP-LISTEN:4444 TCP:192.168.1.99:6666
Adding a –v before the first data channel will allow the intermediary redirecting host to see all the data that flows through the socket. This will also include a date and time for each line of input. This can be particularly useful if you wish to perform extensive logging on this host. Such detailed logging would be ideal if this redirector was in a DMZ acting as a sort of proxy for the socat session.
The final consideration is socat’s default behavior of shutting down the socket after the file transfer is completed. If you wanted to be able to send data and append multiple files to a single file on the server, you need to use the fork option on the server.
$ socat tcp-listen:4444,fork open:test.file,creat,append
This command would allow the server to listen for inbound connections on port 4444. When a connection is made, the data would be written to a file called test.file. When the client disconnects, socat would continue listening on port 4444. Future data would be appended to test.file indefinitely.
Mixing and Matching
Keep in mind that while the examples made use of only the Netcat variant being discussed, at the core they are simple network socket enablers. You can certainly mix and match one version of Netcat running on the host and another running on the server, with a third variant acting as a port redirector in between. In most cases, this should work without problems. You will only have access to any special options on that host, and in some cases a few options may interact in unusual ways. Test the versions you are using together extensively and all should be well.
Because of the large number of Netcat variants and the options they support, I have include a brief feature matrix in Table. This highlights the major file transfer points of interest for each variant, and includes a couple additional notes as well.
Ensuring File Confidentiality
The following is a short summary of the various encryption options for use withNetcat.
- SSL SSL is probably the simplest to implement. It does require a TCP-based Netcat and you must point to the preconfigured local listening SSL port. Stunnel may or may not need to be installed on your particular distribution. OpenSSL is often installed by default.
- SSH In almost all Linux systems SSH will be included in the default install. Like SSL, it does require a TCP-based connection.
- The biggest disadvantage is that SSH is intended for interactive session and requires authentication (i.e., a password) at the time the SSH tunnel is established.
- IPsec IPsec is both the most functional and flexible encryption option, as well as the most complicated. You need to match various security association settings on both systems and multiple files have to be configured in order for it to work.
- IPsec’s primary strengths are the high degree of flexibility in how it is configured and that
- it is protocol independent. You can implement IPsec without making any changes to your specific application or scripts, because the encryption decisions are made by the operating system. This means that IPsec encryption can happen transparently to the application you are encrypting, such as Netcat.
Keeping the files you transfer confidential generally means encryption. If the data
you are sending is not sensitive, encryption might not be a consideration. If confidentiality is a concern, however, an encrypted Netcat-like solution might be just what you need to secure files you send with your home-brewed script or process.
Considering some Netcat variants offer encryption natively, you might wonder why you would want to use a different encryption mechanism. In many cases you may not need anything other than the integrated encryption offered by cryptcat or SBD, but what if those utilities are not a viable choice?
In some environments, using software that is not part of the native distribution is problematic, involving an extensive testing and approval process. Management or security might frown upon adding new software on top of what the distribution originally came with, particularly for mission-critical servers. In these circumstances, it might be easier to just make use of the software that is already installed.
On Windows hosts, the only software that is likely to already be installed is Internet Protocol Security (IPsec) support, because no version of Windows comes with Netcat. Most flavors of Linux, on the other hand, already come with some version of Netcat or a similar product, and frequently some options for encryption.If you find yourself in this situation there are a few options available to you. IPsec, Secure Sockets Layer (SSL), or Secure Shell (SSH) are all viable options for implementing encryption using software that is probably already installed on the hosts in question. Of course in the Windows world, you are probably only going to have IPSec available without resorting to installing additional software.
IPsec also has the advantage that it can encrypt UDP (or Internet Control Message Protocol [ICMP] for that matter) traffic while the other two options can encrypt TCP traffic only.
We will explain how to configure all three of these common solutions. We will
review configuring IPsec on both a Windows host as well as a Linux host.
Using OpenSSH
OpenSSH is an open source implementation of the SSH protocol. Originally it was intended as a secure alternative to other clear text protocols for remote administration, such as Telnet or rshell.SSH includes a port-forwarding option that enables it to function similarly to Stunnel. There are advantages to using OpenSSH over Stunnel, such as the fact that you might already have OpenSSH installed to provide remote access.
In those cases, it would be one less piece of software that you needed to install and configure.Virtually every Linux implementation will come with OpenSSH
installed by default, so you are more likely to already have SSH on your Linux host than you are
SSH
Installing and Configuring Secure Shell SSH requires both an SSH client and an SSH server component.
SSH is the industry standard for remote command line access and most systems come with it as part of the default install. Windows systems are one of the few that do not.There are a variety of products available to bring SSH functionality to Windows, both commercial and free.
There is also a GUI front end for OpenSSH, called PuTTY. Cygwin (is a port of many UNIX tools for Windows and included in this package is an
SSH server). To add even more options, SSHWindows is a free package that installs only the minimum components of the Cygwin package to use SSH, SCP, and Secured File Transfer Protocol (SFTP).
Because the server components and client component operate differently, we will
first walk through setting up the server on a Windows XP system and ensuring that it can successfully accept SSH connections. After the basic functionality of SSH has been verified, we will cover configuring the port forwarding option to tunnel other protocols over the SSH connection.
The SSHWindows package includes both the SSH client files and the SSH server files.
- Download SSHWindows from http://sshwindows.sourceforge.net/ on the client and the server.
- Unzip the file and run the setup utility. Answer the standard prompts and then click Finish.
- At this point the SSH client is ready to be used without the need for any additional configuration. Before you can use the SSH server, however, you must create and edit the \OpenSSH\etc\passwd and \group files.
- If desired, create a separate group on the system to hold users who will have access to SSH, and add the local user accounts to the group for anyone you wish to have access to connect to the SSH server. At the console navigate to the directory where you installed \OpenSSH\bin\.
- Enter the following command on the server to specify which groups can
connect via SSH mkgroup –l >>..\etc\group. This will give all local (–l)
groups permission to connect via SSH.You should open the group file and
edit out the lines corresponding to any groups you do not wish to have access. - Enter the following command on the server to specify any individual accounts that are authorized to connect via SSH mkpasswd –l –u<accountname> >> ..\etc\passwd. You must perform both of these last two steps for SSH to work. If you do not specify the –u <accountname>, all local users will be added to the passwd file.
- Edit the Banner.txt file located in \etc\ to match the banner specified by your
IP security policy.
I:\OpenSSH\bin>ssh sshuser@192.168.1.101 ************ WARNING BANNER HERE ************ sshuser@192.168.1.101’s password: Last login: Sat Jun 24 20:05:22 2006 from 192.168.1.99 Microsoft Windows 2000 [Version 5.00.2195] (C) Copyright 1985-2000 Microsoft Corp. C:\OpenSSH>ipconfig Windows 2000 IP Configuration Ethernet adapter Local Area Connection: Connection-specific DNS Suffix . : rr.com IP Address. . . . . . . . . . . . : 192.168.1.101 Subnet Mask . . . . . . . . . . . : 255.255.255.0 Default Gateway . . . . . . . . . : 192.168.1.1
This is the sample output from sending a file to a host at 192.168.1.101 via SCP.
I:\Internet\OpenSSH\bin>scp sample.txt sshuser@192.168.1.101:/ ************ WARNING BANNER HERE ************ sshuser@192.168.1.101’s password: Could not chdir to home directory /home/SSHuser: No such file or directory sample.txt 100% 1735 1.7KB/s 00:00
While the SSH port in SSHWindows uses standard CMD.exe syntax, the SCP command and SFTP command both use UNIX-style paths. Also of note is that unless it is configured differently, the SSH connection will assume that the directory you installed OpenSSH into is the starting root for client connections.If you get an error of segid: Invalid Argument, this typically means that the permissions are incorrect in the passwd file. The logon account on Windows systems should be 544 instead of 514. The latest installation didn’t seem to have this issue, but it’s not uncommon. A console window should open and the configured logon banner will be displayed. If all is working well, you should then have access to the command prompt on the remote system. With basic SSH working properly and tested, it’s time to move to the port forwarding task.
Configuring OpenSSH Port Forwarding
Working under the assumption that you already have OpenSSH installed and working properly to provide remote command-line access, OpenSSH port forwarding is used much like Stunnel. The SSH client establishes a connection to the SSH server and then listens to local traffic destined for a port you specify. When it receives it, it will send the encrypted data out to an OpenSSH server, which in turn decrypts the data and passes it to the local port the service is listening on. In practical terms this means to set up port forwarding using OpenSSH, perform the following steps.- On the Linux host, run the following command: ssh user@192.168.1.99 –L 5140:192.168.1.99:7140.
- Use Netcat to connect to the server over port 7140.
The primary disadvantage of using SSH for port forwarding is there is no command line means to specify the password. This is an intentional design choice on the part of the SSH developers, in order to increase security. They specifically did not wish to provide a simple means of including passwords in scripts and batch files. This requirement for user interaction makes SSH a better candidate for interactive sessions, rather than service-based connections. As you can see, however, it is very easy to set up on a system that already has SSH configured properly.
Using SSL
SSL can provide a very well documented and simple-to-implement encryption solution for almost any TCP-based communication. SSL is the same well-tested encryption that is commonly used to encrypt Web pages. Stunnel is open-source software, can tunnel TCP communications through an SSL-encrypted session with a minimum of configuration complexity. The end result is the capability to forward a TCP connection through an encrypted tunnel.Configuring Stunnel
Stunnel is perhaps the most widely used tool for encapsulating arbitrary data in an encrypted tunnel.
Stunnel doesn’t contain any cryptographic code, but instead uses external libraries to perform the encryption.In this case, OpenSSL is used to create an encrypted tunnel. To demonstrate the operation of Stunnel on both Windows and Linux, we will be using
- a Linux host as the Netcat client and
- a Windows host as the Netcat server.
- If Stunnel does not come pre-installed on your Linux distribution, then download and install Stunnel from www.stunnel.org.
- If OpenSSL is not already installed on your system, download and install OpenSSL. The source files (to be used on Linux) can be downloaded from www.openssl.org/source/. Pre-compiled binary files (for use on Windows) can be downloaded from www.slproweb.com/products/Win32OpenSSL.html.
- Create a Stunnel configuration file called /etc/stunnel/stunnel.conf. Add the following text to the file and save the file:
- client = yes
[netcat_client]
accept = 127.0.0.1:5140
connect = 192.168.1.99:6140 - This configures Stunnel in client mode (the default is server mode), and tells Stunnel to listen on port 5140 and send the data back out to 192.168.1.99 on port 6140.
- Open a terminal window and start stunnel by typing stunnel. Now you need to configure the server side of the SSL tunnel. Follow these steps to configure Stunnel on the server host.
- Download and install Stunnel from www.stunnel.org.
- Navigate to the directory where you installed Stunnel and edit the stunnel.conf file. Add the following at the end of stunnel.conf and save the changes.
- [netcat_server]
accept = 6140
connect = 7140 - Stunnel will be in server mode by default, and the accept and connect lines simply tell Stunnel to listen on port 6140, and when a connection is made, send the data out to port 7140 on the local host. By not specifying an IP address in our configuration file, Stunnel assumes the local host.
- Navigate to Start > Programs > Stunnel > Run Stunnel.
You should now be able to use Netcat on the client side with the following command;
nc 127.0.0.1 5140
This will tell Netcat to connect to the local host. Stunnel is listening on port 5140 and will redirect the socket across its encrypted tunnel to the recipient. After you start Stunnel, it will place an icon in the system tray. Right-clicking on this icon will open a very small menu. If you select Log on this menu, you can see a running log of the connections that Stunnel has accepted. Because the communication path can be a little confusing, Figure 6.1 shows a graphical representation of the data flow using Stunnel.
After you have verified that you can send and receive your Netcat session over the encrypted Stunnel link, you have only completed the testing phase of the Stunnel implementation. You still need to generate a new server certificate and private key to replace the default key you are using now. Stunnel requires a server certificate and private encryption key for the SSL encryption to function. The installation files come with a default certificate and key, combined into a single file called stunnel.pem.
Because the same certificate is distributed with all of the installation files, using this certificate in a production environment would be insecure because everyone would have your key. You need to generate a new certificate and key file for use on the server (you do not need to do this on the client host). The simplest way of generating the new certificate and key is by using the OpenSSL package.
- OpenSSL should already be installed and working or you wouldn’t have been able to get this far. Use these commands to generate a new server certificate: openssl genrsa –des3 –out server.key 1024. You will be prompted for a pass phrase and it will then generate an initial server key.
- Enter openssl req -new -key server.key -out server.csr to generate a certificate signing request. You will be required to enter the previously assigned pass phrase and answer several prompts with regional information such as your state, country, and company name.
- Enter openssl rsa -in server.key -out server.key to remove the pass phrase from the server key. You will be required to enter the pass phrase to complete this step.
- Enter openssl x509 -req -in server.csr -signkey server.key -out server.crt to generate the server certificate.
- Finally, combine the certificate and key file into a single .pem file that stunnel will use by entering the following command: copy server.crt+server.
key stunnel.pem (for linux cp server.crt stunnel.pem then cat server.key >> stunnel.pem will also work, the files are both plain text files). - Replace the original stunnel.pem with the newly created stunnel.pem certificate in the Stunnel directory.
- Restart Stunnel on the server and verify that everything is still working properly.
The primary downside to using Stunnel is the static nature of the configuration. Because services must be defined in both the client and server stunnel.conf file, you lose the dynamic ability to connect with a single command line.
You must pre-configure the tunnel on the hosts you wish to communicate between ahead of time, before the session can be secured with stunnel.
Using IPsec
Although a little more complicated to configure, IPsec is the industry standard when it comes to setting up Virtual Private Networks (VPNs), which is just another form of encrypted tunnel.IPSec also has a major advantage in that it is protocol independent. IPsec can encrypt TCP, UDP, and even ICMP, basically anything that runs over IP with a few caveats.In the case of encrypting your Netcat session, we only need traffic over that single port encrypted, but IPsec can be used to encrypt all communications between a given host and destination.
The primary downside to using IPsec over one of the previously mentioned solutions is of course the configuration complexity.Additionally, the out-of-box support for modern Windows systems (Windows 2000,
Windows XP, and Windows Server 2003) is better (easier to configure) than that of a Linux host. Although Linux supports IPsec natively, the task of configuring it is made much simpler with the assistance of some third-party configuration utilities. We will demonstrate how to set up IPsec on hosts running Windows XP and Linux.
Configuring IPSec on Windows
For starters, let’s cover some basic Windows IPsec terminology.
- IPsec settings are controlled on a Windows system via the IPsec policy. Only one policy can be applied to a given host at a time.
- The policy is defined using IPsec rules.
- A rule defines what types of traffic to act on and whether the traffic is
- permitted,
- blocked, or
- encrypted.
- The rules also determine how to authenticate the IPsec peer and other encryption settings.
- Filters are used to identify what types of traffic should be processed by the IPSec policy. On some systems this is referred to as defining interesting traffic.
- Security methods are used to define the encryption and hashing algorithms to be used.
- Open the MMC and add the IP Security Policies snap-in to your console if it is not already present.
- Right-click IP Security Policies on Local Computer in the left pane and select Manage IP Filter lists and filter actions.
- On the Manage IP Filter Lists tab, click Add at the bottom. This will bring up the IP Filter List window.
- Enter a name for the filter; in this case we used Netcat.
- Enter a description for the filter.
- Ensure that the Use Add Wizard check box is checked and click Add.
- Click Next on the IP Filter Wizard welcome screen.
- Enter a description of the filter and ensure that the Mirrored option is
checked (it should be by default) and click Next. - On the next window, leave the source address at the default of My IP
Address and click Next. - On the next window, for Destination address, select a specific IP
address, enter the IP address of the Netcat server, and then click Next. - For Select a protocol type, select TCP and click Next.
- For Set the IP protocol port, leave From any port selected and select
To this port. - Enter 4444 (4444 was only selected as an example, you can use a different
port) in the Port box and click Next. - Click Finish, which will take you back to the IP Filter List window. There
should be a new IP Filter in the bottom section of this window. This filter will match against any outbound TCP traffic with a destination IP of the Netcat server and a destination port of 4444. - Click OK to go back to the Manage IP filter lists and filter actions
window. Click Close. We now have our filter defined, which will tell the system what traffic should be processed by the IPsec policy. We now must create that policy. - Right-click IP Security Policies on Local Computer in the left-hand pane and select Create IP Security Policy.
- Click Next to begin the wizard.
- Enter a Name (e.g., Outbound_Netcat) and a Description and click Next.
- In Requests for Secure Communications, leave Activate the default response rule checked and click Next.
- Choose your Authentication method for the default response rule. Active directory is the default and will be the best choice in most circumstances. However, some systems, such as DMZ hosts, may not have domain connectivity, and instead may be standalone servers. In those cases you will need to use certificates or a pre-shared key. A pre-shared key is basically a password and is the weakest of the options available; however, it is also the simplest to implement. For this example, we will use a pre-shared key of “password,” which is not secure but will serve for testing purposes. After making your selection click Next.
- Leave the check box checked to Edit Properties and click Next.
- On the Rules tab, ensure that the Use Add Wizard is not checked and click Add to add an IP filter.
- In the list of pre-made filters you should see the Netcat filter you created earlier. Select the radio button for the filter you created earlier.
- Select the Filter Action tab and select Require Security.
- Click the Authentication Methods tab and click Edit.
- Select the radio button for the authentication method you want to use, and
then click OK. - On the New Rule Properties, click Apply and then OK.
- You will be back at the Security Rules screen; click OK.
- Your Policy should now appear in the list in the right pane of the MMC
window. Right-click this new policy and select Assign. After the policy is
assigned, now all that is left is to configure the IPsec policy on the Netcat server. On the server side, you need to perform a similar configuration; however, there are some implementation details to consider before settling on your configuration. For example, - if all the systems connecting to the server will be using IPsec, you can configure the policy on the server to require IPsec, instead of requesting it.
- If you will have a mixture, with some systems using IPSec secured traffic and some systems using unencrypted communications (internal trusted systems, for example), then the most secure option would be to require security from the individual systems that will be using IPsec, based on their IP address or IP segment.
- In the MMC of the client (not the server), select IP security policies on
local computer in the left pane. - Right-click and select All Tasks > Export Policies.
- Choose a name and location to save the policies and click Save.
- Open the MMC of the server, and if it isn’t already added, add the IP Security Policies snap-in.
- Right-click IP Security Policies on local computer in the left pane, and then select All Tasks > Import Policies....
- Browse to the policies you exported previously and click Open. This will import the exact policy that was configured on the client.
- We will need to make some adjustments to use it on the server, but this process still saves us some time.
- You should see the Netcat policy in the list of policies in the right-hand pane. At this time is should list No under the Policy Assigned column.
- Right-click the Netcat policy and select assign.
Configuring IPSec on Linux
Because IPsec is an industry standard, specifically designed for interoperability between different vendor systems, it is a very popular choice for implementing encryption.
Current Linux kernels have IPsec support built in, and there are other packages that provide their own implementation of IPsec as well.For these examples we will assume you are using the Linux kernel native IPsec support. Follow these steps to configure IPsec on a Linux host.
- If they are not already installed, download and install the IPsec Tools. These tools provide a simplified interface to configure the various IPsec settings.
- Edit the /etc/racoon/psk.txt file. This file holds the pre-shared (aka “secret”) keys. You should use a high-quality password for a production environment. The format of the file is
. In our example we would add the following line to the file and save the new file. 192.168.1.99 password If you wish to change the pre-shared key on the Windows server, edit the IPsec policy by following these steps: - Open the MMC and select IP Security Policies on Local Computer in the left pane.
- Double-click the Inbound_Netcat policy in the right pane.
- Double-click the TCP Syslog security rule and select the Authentication Methods tab.
- Click Edit to change the pre-shared key, and enter the new key.
- Click OK, OK, Apply, and OK to accept the changes and exit the policy configuration windows. The next step is to configure the IPsec policy on the Linux host. With the IPsec tools loaded, this can be done using the setkey utility. The utility can display the current security associations and perform several other configuration changes to your IPsec policy. By creating a setkey configuration file, we will define the security parameters to use. The entire contents of the configuration file are shown in 8.1.
- Create a configuration file (you could name it /etc/racoon/setkey.conf ) and
enter the information shown below: - # Configuration for 192.168.1.105
# Flush the SAD and SPD
flush;
spdflush;
#### ESP SAs using 192 bit long keys (168 + 24 parity) ####
add 192.168.1.105 192.168.1.99 esp 1001
-E 3des-cbc 0x7aeaca3f87d060a12f4a4487d5a5c3355920fae69a96c831;
add 192.168.1.99 192.168.1.105 esp 1001
-E 3des-cbc 0x7aeaca3f87d060a12f4a4487d5a5c3355920fae69a96c831;
#### Security policies ####
spdadd 192.168.1.105 192.168.1.99 any -P out ipsec
esp/transport//require;
spdadd 192.168.1.99 192.168.1.105 any -P in ipsec
esp/transport//require; - Apply the settings in your IPsec policy by entering setkey –f /etc/racoon/setkey.conf.
- Edit your racoon configuration file /etc/racoon/racoon.conf. Racoon is the daemon on Linux that handles your Internet Key Exchange (IKE) functionality. If invoked from the command line with no options, it will automatically be run in daemon mode. For initial testing and setup, we would recommend running it in the foreground, so that you can see the output for troubleshooting purposes. Executing racoon –F will run in the foreground, and adding –d (for debug) will increase the verbosity level to provide even more information. 10.1 shows the complete racoon.conf contents.
- # Racoon IKE daemon configuration file.
# See 'man racoon.conf' for a description of the format and entries.
path include "/etc/racoon";
path pre_shared_key "/etc/racoon/psk.txt";
path certificate "/etc/racoon/certs";
##### IKE PHASE 1
remote 192.168.1.99 {
exchange_mode main;
proposal {
encryption_algorithm 3des;
hash_algorithm sha1;
authentication_method pre_shared_key;
dh_group 2;
}
}
##### IKE PHASE 2
sainfo anonymous
{
lifetime time 1 hour ;
encryption_algorithm 3des, des ;
authentication_algorithm hmac_sha1, hmac_md5 ;
compression_algorithm deflate ;
}
- # Racoon IKE daemon configuration file.
- At a terminal prompt, start racoon using racoon –F or racoon –F –d.
4444. Because IPsec is not protocol dependent, this same type of configuration can easily enable you to encrypt Netcat sessions over TCP or UDP from your Linux system to a Windows system as well. To do this, simply substitute the configuration options of TCP 4444 or UDP 4444, for another port of your choice.
When you first apply all the IPsec settings, you will probably not see traffic immediately. In most cases there will be a short delay while the initial IPsec connection is being established. This time is being spent agreeing on the encryption parameters and exchanging key information prior to the secure communications being able to take place.
Ensuring File Integrity
Of course after you transfer a file, you want to make sure that the file you received is identical to the file which was sent. For very small files any corruption might be obvious, but a visual inspection of a large binary file simply will not do. In these cases you need an automated way to compare the file before and after the transfer. The simplest way to do this is to generate a hash of the file and send that with (before or after) the original file. There are many software packages that can generate a variety of hashes for you, and odds are good you already have at least one of them installed already.A hash is the result of a mathematical computation performed on a set of data. This result, called a hash, hash value, or message digest, is (ideally) unique and reproducible for a given input. In practical terms, this is similar to a digital fingerprint for a set of data. If you were to generate a hash using the entire book “War and Peace” as input, you would get a particular hash value. Anyone else who used the same hash algorithm with “War and Peace” as input would also generate the same hash value.
If any single character were altered in the entire book, such as a period being changed to a comma, the hash value would be different. A hash function is a one-way computation, meaning there is no way to derive the original input from a known hash value. In the case of a file transfer, if the hash that was generated with the file before it was sent matches the hash that is generated with the received file, the two files are identical.
Hashing Tools
There are many utilities for generating a hash value and many different algorithms that are widely used. Some algorithms are more “secure,” in that the odds of two different inputs producing the same hash value are smaller. SHA and Message Digest 5 (MD5) are commonly used algorithms that are considered to be secure enough for most uses. There are many utilities available; we will review a few of the most popular ones below. If you are generating hashes on a Windows server, an excellent utility is fsum from SlavaSoft. Fsum is freeware and can be purchased for commercial use. The license agreement allows fsum to be run on only one computer at a time. We would recommend, as with all free products, that you review and understand the license agreement yourself. Fsum will run on Windows 9x, NT, 2000, and XP and can generate 13 different types of hashes.If you are using Linux you are likely to
- have md5, md5sum, or sha1sum already installed. Both md5sum and sha1sum are special-purpose programs that only generate hash values using their respective algorithms.
- You can also use the OpenSSL suite to generate message digests using a wider variety of algorithms. OpenSSL can generate the following hash types:
- MD2, MD4, MD5,
- rmd160, SHA, and
- SHA1.
The “diff ” utility is found on almost all *nix systems, and modern Windows systems include the compare (comp.exe) utility to do the same thing. There are many different applications to compare two files to choose from if either of these two are not what you are looking for
Using Netcat for Testing
In addition to using Netcat to transfer data for the simple purpose of moving data, it can also be used for troubleshooting and testing. Netcat’s lightweight nature and low overhead means it can serve several useful purposes for network troubleshooting. In a similar fashion to transferring a file, there are more sophisticated troubleshooting tools available. Netcat however is probably more readily available, it’s free, and it has a very small footprint. If you combine these features with a little scripting, Netcat can be a useful tool for troubleshooting and testing your network.Testing Bandwidth
There are times when you need to push a lot of data over a connection for testing purposes. One of the classic tools for doing this is FTP. FTP is a good choice because it is a simple and efficient protocol. Efficient here means that there is little overhead.
This allows FTP to consume a large portion of the available bandwidth. Without any controls, FTP will consume as much of it as possible given the current network conditions. The only downside is you need an FTP client at one end, and an FTP server at the other end to test with. Of course virtually every operating system comes with an FTP client, but a server isn’t always available. If you are testing to the Internet finding an FTP server is trivial, but if you are testing internal links you might not have one available.
The same features that make FTP a good protocol to test with also make Netcat an equally good choice. Netcat is just as lightweight as FTP, possibly even more so. Netcat also has the benefit of being easier to use and in most cases a smaller footprint if you need to “install” Netcat before testing. Given FTP’s reputation as being bandwidth hungry, I ran a test to compare. The results were nearly identical. Both FTP and Netcat utilized 60 percent of my 100Mb Ethernet connection when using redirection at both ends. When I piped the file into Netcat on the linux client using the cat utility, utilization shot up to 98 percent of the available bandwidth. This tells me that Netcat is at least as efficient for testing load on a link as FTP is.
Testing Connectivity
Netcat offers some additional benefits when it comes to testing. Basic connectivity testing is often performed with ICMP’s ping functionality. While adequate in most cases, it does have limitations:
- you cannot test arbitrary ports. Because ping is ICMP based,
- you cannot even test TCP or UDP functionality for that matter.
- You can test layers one through three with pings but no more.
- The fact that Telnet adds Telnet-specific data to the data stream is exactly what makes Netcat a more suitable tool when it comes to interactively testing and probing a given port. With Netcat you can be sure that you will only be sending the data you want to.
- You also cannot transfer most binary files using Telnet. This is because Telnet will interpret some of the binary data as Telnet command strings.
- Telnet also has the major limitation that it is only a client tool. If the server is not running at the far end, but you want to test the firewall rules anyway, Telnet will not help you. Netcat has the option of running as a server as well as a client, so you can run Netcat at both ends and thus test that the firewall is allowing the port through.
Sometimes you know the connection works but you need to test Network Address Translation (NAT) rules. You want to know what IP you are showing up as
when you go through the firewall. If the destination is the Internet, there are many sites to show you your IP, so that part is easy. But if you want to test through some internal NAT or Port Address Translation (PAT) rules, Netcat can be a fast way to do that as well. By increasing the verbosity (–v), Netcat will tell you the IP and port that is being used to connect. Of course the need for these types of testing will probably not arise every day. When they do, however, Netcat is a small, fast tool that can do a lot when put to creative uses.
No comments:
Post a Comment