1 / 48

Web, FTP, and Proxy

Web, FTP, and Proxy. Web Service. Web Service. Three major techniques in WWW (World Wide Web) System HTML – HyperText Markup Language Mark-up the text and define presentation effect by HTML Tags. http://www.w3.org/ HTTP – Hyper-Text Transfer Protocol

rsauter
Download Presentation

Web, FTP, and Proxy

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Web, FTP, and Proxy

  2. Web Service

  3. Web Service • Three major techniques in WWW (World Wide Web) System • HTML – HyperText Markup Language • Mark-up the text and define presentation effect by HTML Tags. • http://www.w3.org/ • HTTP – Hyper-Text Transfer Protocol • Communication method between client and server, both browsers and web servers have to follow this standard. • HTTPS – secured version • URL – Uniform Resource Locator • Describe how to access an object shared on the Internet • Format • Protocol :// [ [ username [ :password ] @ ] hostname [ :port ] ] [ /directory ] [ /filename ]

  4. 1. 以 URL 描述索取的資源位置向 Server 發送要求 3. 從 URL 描述的位置將 HTML 文件取出並回覆給 Client Client Browser Web Server 2. 以 HTTP 協定送出 Request 4. 以 HTTP 協定回覆 Response 5. 接收到 HTML 後由 Browser 解析後根據 HTML 描述定義將資料呈現出來 Web Service – The Client-Server Architecture • Client-server architecture • Web Server: Answer HTTP request • Web Client: Request certain page using URL

  5. Web Service – The HTTP Protocol (1) • HTTP: Hypertext Transfer Protocol • RFCs: (HTTP 1.1) http://www.faqs.org/rfcs/rfc2068.html http://www.faqs.org/rfcs/rfc2616.html (Updated Version) • Useful Reference: http://jmarshall.com/easy/http/ • A network protocol used to deliver virtually all files and other data on the World Wide Web. • HTML files, image files, query results, or anything else. • Client-Server Architecture • A browser is an HTTP client because it sends requests to an HTTP server (Web server), which then sends responses back to the client.

  6. Clients: ※ Send Requests to Servers Action “path or URL” Protocal Actions: GET, POST, HEAD Ex. GET /index.php HTTP/1.1 Headers Header_Name: value Ex. From: someuser@jmarshall.com (blank line) Data … Servers: ※ Respond to the clinets Status: 200: OK 404: Not Found … Ex. HTTP/1.1 200 OK Headers Same as clients Ex. Content-Type: text/html (blank line) Data… Web Service – The HTTP Protocol (2)

  7. nabsd [/home/chwong] -chwong- telnet nabsd.cs.nctu.edu.tw 80 Trying 140.113.17.215... Connected to nabsd.cs.nctu.edu.tw. Escape character is '^]'. GET / HTTP/1.0 Host: nabsd.cs.nctu.edu.tw HTTP/1.0 200 OK Content-Type: text/html Accept-Ranges: bytes ETag: "1897433431" Last-Modified: Tue, 29 May 2007 06:25:04 GMT Content-Length: 94 Date: Tue, 29 May 2007 06:25:06 GMT Server: lighttpd/1.4.15 X-Cache: HIT from nabsd.cs.nctu.edu.tw Via: 1.0 nabsd.cs.nctu.edu.tw:80 (squid/2.6.STABLE13) Connection: close <html> <body> <a href="http://nabsd.cs.nctu.edu.tw/~chwong/docs/"> haha </a> </body> </html> Connection closed by foreign host. action Headers status Headers Data Web Service – The HTTP Protocol (3) • Example:

  8. Web Service – The HTTP Protocol (4) • Get vs. Post (client side) • Get: • Parameters in URL GET http://nabsd.cs.nctu.edu.tw/get.php?a=1&b=3 HTTP/1.1 • No data content • Corresponding in HTML files • Link URL: http://nabsd.cs.nctu.edu.tw/get.php?a=1&b=3 • Using Form: <form method=“GET” action=“get.php”> … </form> • Post: • Parameters in Data Content POST http://nabsd.cs.nctu.edu.tw/post.php HTTP/1.1 • Corresponding in HTML files • Using Form: <form method=“POST” action=“post.php”> … </form>

  9. Web Service – The HTTP Protocol (5) • HTTP Headers: • What HTTP Headers can do? [Ref] http://www.cs.tut.fi/~jkorpela/http.html • Content information (type, date, size, encoding, …) • Cache control • Authentication • URL Redirection • Transmitting cookies • Knowing where client come from • Knowing what software client use • …

  10. Static vs. Dynamic Web Service – Static vs. Dynamic Pages • Static vs. Dynamic Pages • Technologies of Dynamic Web Pages • Client Script Language • JavaScript, Jscript, VBScript • Client Interactive Technology • Java Applet, Flash, XMLHTTP,AJAX • Server Side • CGI • Languages: Perl, ASP, JSP, PHP, C/C++, …etc.

  11. Web Service – Virtual Hosting (1) • Virtual Hosting • Providing services for more than one domain-name (or IP) in one web server. • IP-Based Virtual Hosting vs. Name-Based Virtual Hosting • IP-Base – Several IPs (or ports) • Name-Base – Singe IP, several hostnames • Example (Apache configuration) NameVirtualHost 140.113.17.215 <VirtualHost 140.113.17.215> ServerName nabsd.cs.nctu.edu.tw DocumentRoot "/www/na" </VirtualHost> <VirtualHost 140.113.17.215> ServerName sabsd.cs.nctu.edu.tw DocumentRoot "/www/sa" </VirtualHost> <VirtualHost 140.113.17.215:80> DocumentRoot /www/nabsd ServerName nabsd.cs.nctu.edu.tw </VirtualHost> <VirtualHost 140.113.17.221:80> DocumentRoot /www/tphp ServerName tphp.cs.nctu.edu.tw </VirtualHost>

  12. Web Service – Virtual Hosting (2) Q: How Name-Based Virtual Hosting works? A: It takes use of HTTP Headers. % telnet cswproxy.cs.nctu.edu.tw 80 Trying 140.113.235.111... Connected to cswproxy.cs.nctu.edu.tw. Escape character is '^]'. GET / HTTP/1.0 Host: www.cs.nctu.edu.tw HTTP/1.0 200 OK Date: Tue, 05 Jun 2007 13:50:34 GMT ………… <html> <head> <title>NCTU -- CS</title> <META HTTP-EQUIV="Pragma" CONTENT="no-cache"> <meta http-equiv="refresh" content="0; URL=chinese/doc/index.html"> </head> </html> Connection closed by foreign host. % telnet cswproxy.cs.nctu.edu.tw 80 Trying 140.113.235.111... Connected to cswproxy.cs.nctu.edu.tw. Escape character is '^]'. GET / HTTP/1.0 Host: www.csie.nctu.edu.tw HTTP/1.0 200 OK Date: Tue, 05 Jun 2007 13:51:01 GMT ………… <html> <head> <title>NCTU -- CSIE</title> <meta http-equiv="refresh" content="0; URL=http://www.cs.nctu.edu.tw/"> Connection closed by foreign host.

  13. FTP File Transfer Protocol

  14. FTP • FTP • File Transfer Protocol • Used to transfer data from one computer to another over the internet. • Client-Server Architecture. • Separated control/data connections. • Modes: • Active Mode, Passive Mode • RFCs: • RFC 959 – File Transfer Protocol • RFC 2228 – FTP Security Extensions • RFC 2640 – UTF-8 support for file name

  15. Client Connect to server port 21 using port A. USER #### PASS ******** PORT h1,h2,h3,h4,p1,p2 Send some requestsget return data from p1*256+p2 Quit Server Binding on port 21 Accepts connection from client, output welcome messages. 331 User name okay, need password. 230 User logged in, proceed. 200 PORT Command successful. Binding source port 20, connect to client port p1*256+p2, send data. … FTP – Flow (1)

  16. FTP – Flow (2) • Example • ControlConnection % telnet chonsilab.dyndns.org 21 Trying 140.113.215.86... Connected to chonsilab.dyndns.org. Escape character is '^]'. 220 Serv-U FTP-Server v2.5k for WinSock ready... USER test 331 User name okay, need password. PASS test 230 User logged in, proceed. PORT 140,113,17,215,39,19 200 PORT Command successful. LIST 150 Opening ASCII mode data connection for /bin/ls. 226 Transfer complete. quit 221 Goodbye! Connection closed by foreign host.

  17. FTP – Flow (3) % cat server.pl #!/usr/bin/perl -w package MyPackage; use strict; use base qw(Net::Server::PreFork); MyPackage->run(port => $ARGV[0]); sub process_request { while (<STDIN>) { s/\r?\n$//; print STDERR "$_\n"; } } • Example (contd.) • Retrieving Data • Client must bind the random port %perl server.pl 10003 2007/06/06-13:16:08 MyPackage (type Net::Server::PreFork) starting! pid(4346) Binding to TCP port 10003 on host * Group Not Defined. Defaulting to EGID '1000 110 100 80 0 1000 1000' User Not Defined. Defaulting to EUID '1001' -rwxrwxrwx 1 user group 0 Sep 11 2005 AUTOEXEC.BAT -rwxrwxrwx 1 user group 209 Sep 11 2005 boot.ini -rwxrwxrwx 1 user group 213830 Mar 25 2005 bootfont.bin -rwxrwxrwx 1 user group 0 Sep 11 2005 CONFIG.SYS drwxrwxrwx 1 user group 0 Apr 8 17:30 Documents and Settings -rwxrwxrwx 1 user group 0 Sep 11 2005 IO.SYS -rwxrwxrwx 1 user group 0 Sep 11 2005 MSDOS.SYS -rwxrwxrwx 1 user group 47772 Mar 25 2005 NTDETECT.COM -rwxrwxrwx 1 user group 304752 Mar 25 2005 ntldr drwxrwxrwx 1 user group 0 May 21 23:30 Program Files drwxrwxrwx 1 user group 0 Aug 19 2006 RECYCLER drwxrwxrwx 1 user group 0 Feb 16 2006 System Volume Information drwxrwxrwx 1 user group 0 May 28 16:45 WINDOWS

  18. Commands USER username PASS password LIST Return list of file in current dir. RETR filename Retrieves (gets) file. STOR filename Stores (puts) file onto server. PORT h1,h2,h3,h4,p1,p2 Set to active mode PASV Set to passive mode DELE Remove file on the server. QUIT Return Codes First code 1: Positive Preliminary reply 2: Positive Completion reply 3: Positive Intermediate reply 4: Transient Negative Completion reply 5: Permanent Negative Completion reply Second code 0: The failure was due to a syntax error 1: A reply to a request for information. 2: A reply relating to connection information 3: A reply relating to accounting and authorization. 5: The status of the Server file system FTP – commands, responses

  19. FTP – Active Mode vs. Passive Mode (1) • Active Mode • FTP client bind a random port (>1023) and sends the random port to FTP server using “PORT” command. • When the FTP server initiates the data connection to the FTP client, it binds the source port 20 and connect to the FTP client the random port sent by client. • PORT h1,h2,h3,h4,p1,p2 • Passive Mode • FTP client sends “PASV” command to the server, make the server bind a random port (>1023) and reply the random port back. • When initializing the data connection, the FTP client connect to the FTP Server the random port, get data from that port. • PASV  Server reply: 227 Entering Passive Mode (h1,h2,h3,h4,p1,p2) ※ IP:port (6bytes)  h1,h2,h3,h4,p1,p2 Ex. 140.113.17.215:45678  140,113,17,215,178,110

  20. FTP – Active Mode vs. Passive Mode (2) Active mode Passive mode

  21. FTP – When FTP meets NAT/Firewall (1) • Firewall behavior • Generally, the NAT/Firewall permits all outgoing connection from internal network, and denies all incoming connection from external network. • Problem when FTP meets NAT/Firewall • Due to the separated command/data connection, the data connections are easily blocked by the NAT/Firewall. • Problem Cases: • Active mode, NAT/Firewall on client side. • Passive mode can solve this problem. • Passive mode, NAT/Firewall on server side. • Active mode can solve this problem. • Both client side and server side have NAT/Firewall • The real problem.

  22. NAT/Firewall NAT/Firewall Client Server Client Server PORT IP, port Y PASV reply IP, port Z Connect to port YBLOCKED Connect to port Z PASS Active Mode Passive Mode FTP – When FTP meets NAT/Firewall (2) • Active mode, NAT/Firewall on client side. • Passive mode can solve this problem.

  23. NAT/Firewall NAT/Firewall Client Server Client Server PASV PORT IP, port Y reply IP, port Z Connect to port Z BLOCKED Connect to port YPASS Passive Mode Active Mode FTP – When FTP meets NAT/Firewall (3) • Passive mode, NAT/Firewall on Server side. • Active mode can solve this problem.

  24. NAT/Firewall NAT/Firewall NAT/Firewall NAT/Firewall Client Server Client Server PORT IP, port Y PASV reply IP, port Z Connect to port YBLOCKED Connect to port Z BLOCKED Passive Mode Active Mode FTP – When FTP meets NAT/Firewall (4) • Real Problem: Firewall on both sides. • Solution: ftp-proxy running on NAT/Firewall

  25. FTP – Security • Security concern • As we seen, FTP connections (both command and data) are transmitted in clear text. • What if somebody sniffing the network? • We need encryption. • Solutions • FTP over SSH • So called secure-FTP. • Both commands and data are encrypted while transmitting. • Poor performance. • FTP over TLS • Only commands are encrypted while transmitting. • Better performance.

  26. FTP – Pure-FTPd (1) • Introduction • A small, easy to set up, fast and secure FTP server • Support chroot • Restrictions on clients, and system-wide. • Verbose logging with syslog • Anonymous FTP with more restrictions • Virtual Users, and Unix authentication • FXP (File eXchange Protocol) • FTP over TLS • UTF-8 support for file names

  27. FTP – Pure-FTPd (2) Installation Ports: /usr/ports/ftp/pure-ftpd Options

  28. FTP – Pure-FTPd (3) Other options WITH_CERTFILE for TLS Default: /etc/ssl/private/pure-ftpd.pem WITH_LANG Change the language of output messages Startup: Add pureftpd_enable=“YES” into /etc/rc.conf

  29. FTP – Pure-FTPd Configurations(1) Configurations: File: /usr/local/etc/pure-ftpd.conf Documents Configuration sample: /usr/local/etc/pure-ftpd.conf.sample All options are explained clearly in this file. Other documents See /usr/local/share/doc/pure-ftpd nabsd [/usr/local/share/doc/pure-ftpd] -chwong- ls AUTHORS README README.MySQL THANKS CONTACT README.Authentication-Modules README.Netfilter pure-ftpd.png COPYING README.Configuration-File README.PGSQL pureftpd.schema HISTORY README.Contrib README.TLS NEWS README.LDAP README.Virtual-Users

  30. FTP – Pure-FTPd Configurations(2) # Cage in every user in his home directory ChrootEveryone yes # If the previous option is set to "no", members of the following group # won't be caged. Others will be. If you don't want chroot()ing anyone, # just comment out ChrootEveryone and TrustedGID. TrustedGID 0 # PureDB user database (see README.Virtual-Users) PureDB /etc/pureftpd.pdb # If you want simple Unix (/etc/passwd) authentication, uncomment this UnixAuthentication yes # Port range for passive connections replies. - for firewalling. PassivePortRange 30000 50000 # This option can accept three values : # 0 : disable SSL/TLS encryption layer (default). # 1 : accept both traditional and encrypted sessions. # 2 : refuse connections that don't use SSL/TLS security mechanisms, # including anonymous sessions. # Do _not_ uncomment this blindly. Be sure that : # 1) Your server has been compiled with SSL/TLS support (--with-tls), # 2) A valid certificate is in place, # 3) Only compatible clients will log in. TLS 2 # UTF-8 support for file names (RFC 2640) # Define charset of the server filesystem and optionnally the default charset # for remote clients if they don't use UTF-8. # Works only if pure-ftpd has been compiled with --with-rfc2640 FileSystemCharset big5 # ClientCharset big5

  31. FTP – Pure-FTPd Problem Shooting Logs Location In default, syslogd keeps ftp logs in /var/log/xferlog Most frequent problem pure-ftpd: (?@?) [ERROR] Unable to find the 'ftp' account It’s ok, but you may need it for Virtual FTP Account. pure-ftpd: (?@?) [ERROR] Sorry, but that file doesn't exist: [/etc/ssl/private/pure-ftpd.pem] If you set TLS = 2, then this file is needed. How to generate a pure-ftpd.pem? See README.TLS

  32. FTP – Pure-FTPd Tools pure-* pure-ftpwho List information of users who use the FTP server now. pure-pw To create Virtual Users using PureDB man pure-pw See README.Virtual-Users

  33. FTP – PF: Issues with FTP (1) Reference: http://www.openbsd.org/faq/pf/ftp.html FTP Client Behind the Firewall Problem Clients cannot use active mode Use ftp-proxy Use inetd to start ftp-proxy man ftp-proxy In pf.conf nat-anchor “ftp-proxy/*” rdr-anchor “ftp-proxy/*” rdr on $int_if proto tcp from any to any port 21 -> 127.0.0.1 \ port 8021 anchor “ftp-proxy/*”

  34. FTP – PF: Issues with FTP (2) PF “Self-Protecting” an FTP Server Problem Clients cannot use passive mode Open holes so that clients can connect into the data channel In pf.conf pass in on $ext_if proto tcp from any to any port 21 keep state pass in on $ext_if proto tcp from any to any port > 49151 keep state

  35. FTP – PF: Issues with FTP (3) FTP Server Protected by an External PF Firewall Running NAT Problem Clients cannot use passive mode Use ftp-proxy Need some flags of ftp-proxy man ftp-proxy In pf.conf nat-anchor “ftp-proxy/*” nat on $ext_if inet from $int_if -> ($ext_if) rdr-anchor “ftp-proxy/*” pass in on $ext_if inet proto tcp to $ext_ip port 21 flags S/SA keep state pass out on $int_if inet proto tcp to $ftp_ip port 21 user proxy flags S/SA keep state anchor “ftp-proxy/*”

  36. FTP – More Tools /usr/ports/ftp/pftpx Another ftp proxy daemon /usr/ports/ftp/lftp A powerful functional client Support TLS FileZilla An FTP Client for Windows Support TLS

  37. Proxy

  38. ProxyServer Request Request client Reply Reply Original Server Request Reply(using cached result) client Proxy • Proxy • A proxy server is a server which services the requests of its clients by: • Making requests to other servers • Caching some results for further same requests • Goals: • Performance • Stability • Central Control • …etc. • Roles: • Forward Proxy • Reverse Proxy • Targets • Web/FTP Pages • TCP/IP Connections • …etc.

  39. ProxyServer Request Request client Reply Reply Original Server Request Reply(using cached result) client Proxy – The Forward Proxy • Forward Proxy • Proxy the outgoing requests, for the reason of • Bandwidth saving • Performance • Central control • When objects requested are • In cache, return the cached objects • Otherwise, proxy server requests object from origin server, then cache it and return to client

  40. Reverse ProxyServer Request Internet Reply Server1 client Request Reply client Server1 Proxy – The Reverse Proxy • Reverse Proxy • Proxy the incoming requests, for the reason of • Reducing Server Load (by caching) • Load Balance • Fault Tolerant • Reverse proxy acts as the original server, accept incoming requests, reply corresponding result. SEAMLESS for clients!

  41. Proxy – SQUID • A web proxy server & cache daemon. • Supports HTTP, FTP • Limited support for TLS, SSL, Gopher, HTTPS • Latest stable version: 2.6-STABLE13, 2007/5/11 • Port install: /usr/ports/www/squid • Startup: • /etc/rc.conf • squid_enable="YES" • squid_config="/usr/local/etc/squid/squid.conf" • squid_user="squid" • /usr/local/etc/rc.d/squid start • Configuration Sample/Documents: • /usr/local/etc/squid/squid.conf.default

  42. Proxy – SQUID Configuration (1) • Listen Port • Service Port • http_port 3128 • Neighbored Communication • icp_port 3130 • Logs • access_log • access_log /var/log/squid/access.log squid • cache_log • cache_log /var/log/squid/cache.log • cache_store_log • cache_store_log /var/log/squid/store.log

  43. Proxy – SQUID Configuration (2) • Access Control • acl – define an access control list • Format: acl acl-nameacl-typedata acl all src 0.0.0.0/0.0.0.0 acl NCTU srcdomain .nctu.edu.tw acl YAHOO dstdomain .yahoo.com acl allowhost src “/usr/local/etc/squid.squid.allow” • http_access – define the control rule • Format: http_access allow|deny acl-name http_access allow NCTU http_access allow allowhost http_access deny all

  44. Proxy – SQUID Configuration (3) • Proxy Relationship • Protocol: ICP (Internet Cache Protocol)RFC 2186 2187, using UDP • Related Configuration • cache_peer hostname type http_port icp_port [options] • cache_peer_domain cache-host domain [domain …] • cache_peer_access cache-host allow|deny acl-name

  45. Proxy – SQUID Configuration (4) • Cache Control • cache_mem 256 MB • cache_dir ufs /usr/local/squid/cache 100 16 256 • cache_swap_low 93 • cache_swap_high 98 • maximum_object_size 4096 KB • maximum_object_size_in_memory 8 KB

  46. Proxy – SQUID Configuration (5) • Sample: Proxy Configuration http_port 3128 icp_port 3130 cache_mem 32 MB cache_dir ufs /usr/local/squid/cache 100 16 256 access_log /var/log/squid/access.log squid cache_log /var/log/squid/cache.log cache_store_log /var/log/squid/store.log pid_filename /usr/local/squid/logs/squid.pid visible_hostname nabsd.cs.nctu.edu.tw acl allowhosts src "/usr/local/etc/squid/squid.allow“ http_access allow allowhosts http_access deny all

  47. Proxy – SQUID Configuration (6) • Sample: Reverse Proxy Configuration http_port 80 vhost icp_port 3130 cache_mem 32 MB cache_dir ufs /usr/local/squid/cache 100 16 256 access_log /var/log/squid/access.log squid cache_log /var/log/squid/cache.log cache_store_log /var/log/squid/store.log pid_filename /usr/local/squid/logs/squid.pid visible_hostname nabsd.cs.nctu.edu.tw url_rewrite_program /usr/local/squid/bin/redirect.pl acl cswww dstdomain csws1 csws2 http_access allow all cswww always_direct allow cswww

  48. Proxy – SQUID Configuration (7) % cat /usr/local/squid/bin/redirect.pl #!/usr/bin/perl $|=1; # use non-blocking I/O while(<STDIN>){ if (/^http:\/\/www\.cs\.nctu\.edu\.tw\/([^\s]*)/) { my $ran = int(rand(2)+1); print "http://csws$ran.cs.nctu.edu.tw/$1\n"; next; } print "\n"; }

More Related