1 / 39

Wireless Video Conferencing

Learn about setting up wireless video conferencing systems, communications protocols, video compression, and picture-in-picture display.

robertlgarcia
Download Presentation

Wireless Video Conferencing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Wireless Video Conferencing EECS150 Spring 2007 - Lab Lecture #10 Neil Warren Allen Lee EECS150 Lab Lecture #8

  2. Checkpoint #3 Information(1) • Check off Requirement: • #1) Be able to set 2 sets different source and destination addresses on 2 boards and communicate. (CP4 video streaming) • #2) Be able to set both sides’ destination address to 0xFF and a unique source address on each and have communications occur. (CP4 Handshake) EECS150 Lab Lecture #8

  3. Checkpoint #3 Information • If you’re behind, you can forfeit checkpoint #3 and receive a black box. • You will receive a 0 for CP3 but will be able to move on to CP4 which is worth many more points. • Talk to Neil if you want this option. • You lose the points before you get the BB. EECS150 Lab Lecture #8

  4. Welcome to Checkpoint 4 • Combine CP2 and CP3 to make two-way wireless video conferencing system • Note: Specification has changed to make your life much easier – based on questions from students so far. Make sure you work with the non-draft version of the Checkpoint4.doc EECS150 Lab Lecture #8

  5. Overview • Revisit CP2 and CP3 • Communications protocol • Video compression • Picture-in-picture display EECS150 Lab Lecture #8

  6. Recall from Checkpoint 2 EECS150 Lab Lecture #8

  7. Checkpoint 4 Datapath EECS150 Lab Lecture #8

  8. Our Advice (1) • Separate the work between you and your partner to make the best use of your time. • Communications System • Graphics System • Agree on an interface and make sure you conform to it. EECS150 Lab Lecture #8

  9. Our Advice (2) • Debug Independently! • Communications System: • Assume that the graphics system works perfectly (conforms to your interface). • Hardcode in data and requests using timers and debug the system by using the packet sniffer. • Graphics System: • Assume that the communications system works perfectly. • Hardcode transmission inputs, and whatever interface method you use for send/receive acknowledgements. • Much easier to debug each piece, then just put the 2 working pieces together. • If you conform to your interface it should work! EECS150 Lab Lecture #8

  10. Communications Module (1) • Responsibilities • Maintain communications link. • Lost packet robustness • Provide cross-compatibility • Conform to communications protocol • Encode and Decode packets and pass data to graphics engine • Handle the initial handshake between master and slave. EECS150 Lab Lecture #8

  11. Communications Module (2) • Recommended Pieces: • State Machine • Keeps track of the current state of the communications module. • Handles handshaking and properly re-synchronizing the communications link after lost packet. • Sends commands to Packet Encoder, receives updates from Packet Decoder • Packet Encoder • Generate payload data for each type of packet and handle handshake with transceiver. • Take commands from state machine. • Packet Decoder • Passes data to the graphics engine and gives status updates to the state machine. EECS150 Lab Lecture #8

  12. Communications Module (3) • Initial Handshaking • Purpose: • To allow any 2 boxes without knowledge of the other’s address to communicate. • To set up the timing of master and slave send and receives. Note: At the beginning both the master and slave begin with their destination addresses being 0xFF and their source addresses being unique EECS150 Lab Lecture #8

  13. Communications Module (4) • Handshake Method: • Master broadcasts “Master-Seek” packets once every 2.4ms. When a slave receives a “Master-Seek” packet it sends a “Slave-ReqAck” • When the master receives the “Slave-ReqAck” packet it changes its destination address to the source address of the slave and then transmits a “Master-Ready” packet with the new destination address. • When the slave receives a “Master-Ready” packet the slave sends a “Slave-Ready” packet and moves into a state where it can receive active video packets from the master. The “Slave-Ready” packet should be sent with its destination address as the source of the address of the master. • When the master receives the “Slave-Ready” ready packet, it sends its first packet of active video data. • Now the communication should alternate between the master and slave transmitting active video data. EECS150 Lab Lecture #8

  14. Communications Module (5) • Communications Protocol • Purpose: • To allow cross-compatibility between different implementations. • Abstracts away the communications aspect of the project for the graphics module. • The 8-bit header designates the type of packet. • The 8-bit line number indicates which line the respective active video data corresponds to. • The 80-bit data contains the AV data for 32 pixels. (16 pixel pairs) • 8 bit headers and line numbers make it easy to view on the packet sniffer! EECS150 Lab Lecture #8

  15. Communications Module (6) EECS150 Lab Lecture #8

  16. Communications Module (7) • Time Outs – let you know if a communication loss has occurred. • After the expiration of 10ms from the last AV packet transmission, if the slave not does receive a packet it should return to a state where it can receive AV packets from the master. • If this occurs 25 consecutive times without ever receiving an AV packet, then a communication loss is declared and the system must be reset. A message should appear on screen indicating a communication loss has occurred. • After the expiration of 10ms from the last AV packet received, if the master does not receive a packet it should send its next AV packet. • If this occurs 25 consecutive times without ever receiving an AV packet, then a communication loss is declared and the system must be reset. A message should appear on screen indicating a communication loss has occurred. EECS150 Lab Lecture #8

  17. Communications Module (8) • Tips: • Make use of the packet sniffer to determine what is *actually* flying through the air. • The communications protocol was re-designed specifically for this purpose. EECS150 Lab Lecture #8

  18. Video Compression (1) • Wireless transfer is the bottleneck • Uncompressed video: • 507x720x16 = 5,840,640 bits per frame • That’s ~175 Mbits per second! • Theoretical max bandwidth for wireless is 256kbps, so ~22.8 seconds per frame. EECS150 Lab Lecture #8

  19. Video Compression (2) • Simple lossy compression algorithm • Resolution: 176x127 • 5 bits grayscale per pixel pair EECS150 Lab Lecture #8

  20. Video Compression (3) • Compressed Video: • 176x127x5/2 = 55,880 bits per frame • Approximately 105:1 compression • At theoretical max wireless bandwidth, that’s 4.5 frames per second EECS150 Lab Lecture #8

  21. Video Compression (4) • Vertical compression • Sample every other odd active line • (254 active odd lines)/2 = 127 lines • Horizontal compression • Sample every fourth pixel pair (except for the last eight) • (360-8)/4 = 88 pixel pairs = 176 pixels • For each sampled pixel pair, keep only the upper five bits of the average luminance EECS150 Lab Lecture #8

  22. Video Compression (5) • Why 176x127 instead of 180x127? • Good question. There’s a good answer, figure it out! EECS150 Lab Lecture #8

  23. Video Decompression • Reverse of compression, but lossy due to lack of chrominance • 5-bit grayscale represents a pixel pair • Decompresses to 32 bits, with appropriate padding for chrominance values EECS150 Lab Lecture #8

  24. Writing Compressed Frames • Set aside an area in SDRAM for sending and receiving compressed frames • Update the stored frame to send only after the entire frame has been sent • It is okay to lose a local frame to store a compressed one EECS150 Lab Lecture #8

  25. Writing Compressed Frames • Sequence for handling compressed frames: • Wait for SDRAM and ChipCon initialization • Store a compressed frame • Tell transmitter FSM that frame in SDRAM is ready • Wait as compressed frame is sent • Receive signal from transmitter that last packet for frame has been sent • Once entire compressed frame is sent, store a new compressed frame EECS150 Lab Lecture #8

  26. Writing Compressed Frames • Two options for implementing compressed video controller: • Merge with VD processor, using the same FIFO • Advantage: Arbiter sees one write port for local video • Drawback: Need to cross clock boundaries • Separate module with own FIFO • Advantage: May not need to cross clock boundaries • Drawback: Arbiter sees two write ports for local video • Pick the one that makes more sense to you EECS150 Lab Lecture #8

  27. Dual-Clock Communication • Modules running on the camera clock and the system clock cannot safely communicate with direct control signals. • If your modules need to communicate but are on different clocks, you must either implement a safe handshaking protocol or use the memory element we provide. • For your convenience, we’ve included a dual-clocked one-bit-wide FIFO to safely exchange control signals, but you don’t have to use it. EECS150 Lab Lecture #8

  28. Sender and Receiver FSMs • Sender/Transmitter • Waits for frame to be available in SDRAM • Reads from SDRAM and packages data into a packet • Requests a new frame once last packet for frame is sent • Receiver • Decodes the compressed data • Writes received data into SDRAM EECS150 Lab Lecture #8

  29. Displaying Compressed Video EECS150 Lab Lecture #8

  30. Displaying Compressed Video • Constantly display compressed frames to avoid SDRAM decay • This way, you don’t have to auto-refresh! EECS150 Lab Lecture #8

  31. Multi-Port Arbiter • At least two more ports are needed for the sender and receiver FSMs. • A fifth port may be needed depending on how you handle writing compressed frames. EECS150 Lab Lecture #8

  32. Tips, Hints, & Common Pitfalls • Start early! • If you’re already done with CP3, start now! • We are not providing any Verilog besides the optional one-bit-wide memory element. • Run the TA solution to see what the final result should look like. EECS150 Lab Lecture #8

  33. Tips, Hints, & Common Pitfalls • Split up the project into graphics and communications. • These parts are fairly independent so you and your partner can work in parallel. • Do not combine until you’ve verified each part works independently. • It can take up to 25 minutes to push to board after combining! EECS150 Lab Lecture #8

  34. Tips, Hints, & Common Pitfalls • Work Incrementally • Display a sectioned screen • Display only compressed video at full frame rate • Simultaneously display local compressed and uncompressed video at full frame rate • Display uncompressed video at full frame rate and local compressed at 1 FPS • Test sender and receiver FSMs by sending local “packets” to yourself EECS150 Lab Lecture #8

  35. Tips, Hints, & Common Pitfalls • Be careful how you handle the half-full packet (packet 6): • Make sure the receiver knows which half of the payload to look at for the video data EECS150 Lab Lecture #8

  36. Tips, Hints, & Common Pitfalls • 2 Clocks with phase offset: • Camera Clock != Normal Clock • Use the 1-bit wide dual-clocked fifo we provide to cross clock boundaries. EECS150 Lab Lecture #8

  37. Requirements • Must be TA Solution Compatible. • EC can be non-compatible, but you must still develop a compatible solution first. • 1 bit file. Both master and slave systems should contained on 1 bit file. • Onscreen prompts. Should display to the user the communications status (see TA solution for more info) • Some method of outputting the (max) time between new frames. EECS150 Lab Lecture #8

  38. Extra Credit • Maximum 20% • Early check off 10% • Some Ideas (EC% TBD): • Faster frame rate • Larger packet sizes • Faster transmission • Non-full duplex communications • UI options • Menus • Video output modifications (change contrast with buttons, etc.) • Use N64 controller for extra buttons to add functionality • We will provide a black box to interface • Open-ended! Come up with something special, cool, clever, unique, and/or challenging. EECS150 Lab Lecture #8

  39. Questions? EECS150 Lab Lecture #8

More Related