1 / 31

Malware Analysis and Instrumentation

Malware Analysis and Instrumentation. Andrew Bernat and Kevin Roundy. Paradyn Project. Paradyn / Dyninst Week Madison, Wisconsin May 2-4, 2011. Forensic analysts need help. 90% of malware resists analysis [1] Malware attacks cost billions of dollars annually [2]

glennis
Download Presentation

Malware Analysis and Instrumentation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Malware Analysis and Instrumentation • Andrew Bernat and Kevin Roundy Paradyn Project Paradyn / Dyninst Week Madison, Wisconsin May 2-4, 2011

  2. Forensic analysts need help • 90% of malware resists analysis[1] • Malware attacks cost billions of dollars annually[2] • 65% of users feel effect of cyber crime[3] • 69% cybercrimes are resolved[3] • 28 days on average to resolve a cybercrime[3] Malware Binary 7a 77 0e 20 e9 3d e0 09 e8 68 c0 45 be 79 5e 80 89 08 27 c0 73 1c 88 48 6a d8 6a d0 56 4b fe 92 57 af 40 0c b6 f2 64 32 f5 07 b6 66 21 0c 85 a5 94 2b 20 fd 5b 95 Malware Analysis and Instrumentation [1] McAfee. 2008 [2] Computer Economics. 2007 [3] Norton. 2010

  3. Forensic analysts need help The needed toolbox • Binary code identification • Control- and data-flow analysis • Instrumentation • Effectiveness on malware Malware Binary 7a 77 0e 20 e9 3d e0 09 e8 68 c0 45 be 79 5e 80 89 08 27 c0 73 1c 88 48 6a d8 6a d0 56 4b fe 92 57 af 40 0c b6 f2 64 32 f5 07 b6 66 21 0c 85 a5 94 2b 20 fd 5b 95 Malware Analysis and Instrumentation

  4. Dyninst is a toolbox for analysts library injection function replace- ment loop, block, function, instruction instrument- ation symbol table reading, writing forward & backward slices machine language parsing CFG loop analysis call stack walking Dyninst Dyninst binary rewriting program binary 7a 77 0e 20 e9 3d e0 09 e8 68 c0 45 be 79 5e 80 89 08 27 c0 73 1c 88 48 6a d8 6a d0 56 4b fe 92 57 af 40 0c b6 f2 64 32 f5 07 b6 66 21 Control flow analyzer Data flow analyzer Instrumenter process control Malware Analysis and Instrumentation

  5. Dyninst is a toolbox for analysts Analysis tool library injection function replace- ment loop, block, function, instruction instrument- ation symbol table reading, writing Mutator forward & backward slices machine language parsing CFG CFG • Specifies instrumentation • Gets callbacks for runtime events • Builds high-level analysis loop analysis call stack walking Dyninst Dyninst binary rewriting program binary 7a 77 0e 20 e9 3d e0 09 e8 68 c0 45 be 79 5e 80 89 08 27 c0 73 1c 88 48 6a d8 6a d0 56 4b fe 92 57 af 40 0c b6 f2 64 32 f5 07 b6 66 21 Control flow analyzer Data flow analyzer Instrumenter process control Malware Analysis and Instrumentation

  6. Dyninst is a toolbox for analysts Code snippets printf(…) getTarget(insn) counter++ if (pred) callback(…) Code visualizations Analysis tool Analysis of network communications Mutator CFG • Specifies instrumentation • Gets callbacks for runtime events • Builds high-level analysis Time bomb detection and analysis Identification of stolen data Dyninst program binary 7a 77 0e 20 e9 3d e0 09 e8 68 c0 45 be 79 5e 80 89 08 27 c0 73 1c 88 48 6a d8 6a d0 56 4b fe 92 57 af 40 0c b6 f2 64 32 f5 07 b6 66 21 Reports on anti-analysis techniques Control flow analyzer Data flow analyzer Instrumenter Malware Analysis and Instrumentation

  7. Dyninst on malware Code snippets printf(…) getTarget(insn) counter++ if (pred) callback(…) Code visualizations Code visualizations Malware defeats static analysis & is sensitive to instrument-ation Analysis tool Analysis of network communications Analysis of network communications Mutator CFG • Specifies instrumentation • Gets callbacks for runtime events • Builds high-level analysis Time bomb detection and analysis Time bomb detection and analysis Identification of stolen data Identification of stolen data Dyninst malware binary 7a 77 0e 20 e9 3d e0 09 e8 68 c0 45 be 79 5e 80 89 08 27 c0 73 1c 88 48 6a d8 6a d0 56 4b fe 92 57 af 40 0c b6 f2 64 32 f5 07 b6 66 21 Reports on anti-analysis techniques Reports on anti-analysis techniques Control flow analyzer Data flow analyzer Instrumenter Malware Analysis and Instrumentation

  8. Dyninst on malware Code snippets printf(…) getTarget(insn) counter++ if (pred) callback(…) Code visualizations Malware defeats static analysis & is sensitive to instrument-ation Analysis tool Analysis of network communications Mutator CFG CFG • Specifies instrumentation • Gets callbacks for runtime events • Builds high-level analysis Time bomb detection and analysis Identification of stolen data SR- Dyninst Dyninst malware binary 7a 77 0e 20 e9 3d e0 09 e8 68 c0 45 be 79 5e 80 89 08 27 c0 73 1c 88 48 6a d8 6a d0 56 4b fe 92 57 af 40 0c b6 f2 64 32 f5 07 b6 66 21 static-dynamic analysis Sensitivity Resistant Instrumenter Reports on anti-analysis techniques Control flow analyzer Data flow analyzer Instrument-er Control flow analyzer Data flow analyzer CFG Malware Analysis and Instrumentation

  9. Outline Anti-analysis tricks Hybrid static-dynamic analysis Sensitivity resistance Results Anti H.A. S.R. Res. 9 Malware Analysis and Instrumentation

  10. Anti-analysis tricks Anti Obfuscated control flow Obfuscated control flow indirect control flow, stack tampering, overlapping code, signal-based ctrl flow Unpacked code Unpacked code all-at-once, block-, loop-, function-at-a-time, to empty or allocated space Anti-analysis Overwritten code single operand or opcode, whole instruction, function, code section, buffer Overwritten code PC-sensitive code PC-sensitive code call-pop pairs, return-address manipulation, call-stack tampering & probing Anti-patching Anti-patching checksum whole regions, probe for patches, use code as data, move stack ptr Anti-instrumentation Address-space probing Address-space probing scans & probes of locations that should be un-allocated Malware Analysis and Instrumentation

  11. Obfuscated control flow Anti 40d002 storm worm Entry Point obfuscated control flow obfuscated control flow unpacked code overwritten code pc-sensitive code anti-patching address-space probing Malware Analysis and Instrumentation

  12. Unpacked code Anti storm worm Entry Point obfuscated control flow obfuscated control flow unpacked code overwritten code 7a 77 0e 20 e9 3d e0 09 e8 68 c0 45 be 79 5e 80 89 08 27 c0 73 1c 88 48 6a d8 6a d0 56 4b fe 92 57 af 40 0c b6 f2 64 32 f5 07 b6 66 21 0c 85 a5 94 2b 20 fd 5b 95 e7 c2 16 90 14 8a 14 26 60 d9 83 a1 37 1b 2f b9 51 84 02 1c 22 8e 63 01 7a 77 0e 20 e9 3d e0 09 e8 68 c0 45 be 79 5e 80 89 08 27 c0 73 1c 88 48 6a d8 6a d0 56 4b fe 92 57 af 40 0c b6 f2 64 32 f5 07 b6 66 21 0c 85 a5 94 2b 20 fd 5b 95 e7 c2 16 90 14 8a 14 26 60 d9 83 a1 37 1b 2f b9 51 84 02 1c 22 8e 63 01 pc-sensitive code anti-patching address-space probing 12 Malware Analysis and Instrumentation

  13. Overwritten code Anti Entry Point Upack packer obfuscated control flow obfuscated control flow unpacked code overwritten code 7a 77 0e 20 e9 3d e0 09 e8 68 c0 45 be 79 5e 80 89 08 27 c0 73 1c 88 48 6a d8 6a d0 56 4b fe 92 57 af 40 0c b6 f2 64 32 f5 07 b6 66 21 0c 85 a5 94 2b 20 fd 79 5e 80 89 08 27 c0 73 1c 88 48 6a d8 5b 95 e7 c2 16 90 14 8a 14 26 60 d9 83 a1 37 1b 2f b9 51 84 02 1c 22 8e 63 01 pc-sensitive code anti-patching address-space probing 13 Malware Analysis and Instrumentation

  14. PC Sensitive code Anti e.g., ASProtect Use call to get current PC Local Data Access call data Pop PC into register obfuscated control flow obfuscated control flow pop esi add esi, eax mov ebx, ptr[esi] unpacked code Construct pointer and dereference overwritten code pc-sensitive code anti-patching address-space probing 14 Malware Analysis and Instrumentation

  15. Anti-patching Anti Checksumming detects instrumentation [Aucsmith 96] e.g., PECompact checksum routine protected code xor eax, eax calculate checksum of protected region add eax, ptr[ebx] add 4, ebx cmp ebx, 0x41000 jne .loop obfuscated control flow compare to expected value cmp eax, .chksum jne .fail jmp unpacked code overwritten code pass fail fail instrument-ationis detected pc-sensitive code anti-patching address-space probing 15 Malware Analysis and Instrumentation

  16. Address-space probing Anti code Memory Scan int *ptr = 0; data segv_handler() { ptr += PAGESIZE; goto RESTART: } code instrumentation sigaction(SIGSEGV, segv_handler); while(1) { RESTART: *ptr; ptr += PAGESIZE; } obfuscated control flow obfuscated control flow unpacked code overwritten code pc-sensitive code anti-patching address-space probing 16 Malware Analysis and Instrumentation

  17. Code discovery algorithm H.A. Hybrid algorithm: Parse from known entry points Instrument control flow that may lead to new code Resume execution ? ? instrument overwrite exception CALL ptr[eax] DIV eax, 0 Malware Analysis and Instrumentation

  18. Code discovery algorithm H.A. Hybrid algorithm: Parse from known entry points Instrument control flow that may lead to new code Resume execution ? ? instrument overwrite exception CALL ptr[eax] DIV eax, 0 Malware Analysis and Instrumentation

  19. Code discovery algorithm H.A. Hybrid algorithm: Parse from known entry points Instrument control flow that may lead to new code Resume execution ? ? instrument overwrite exception CALL ptr[eax] DIV eax, 0 Malware Analysis and Instrumentation

  20. Code discovery algorithm H.A. Hybrid algorithm: Parse from known entry points Instrument control flow that may lead to new code Resume execution ? ? instrument overwrite exception CALL ptr[eax] DIV eax, 0 Malware Analysis and Instrumentation

  21. Code discovery algorithm H.A. Hybrid algorithm: Parse from known entry points Instrument control flow that may lead to new code Resume execution ? instrument overwrite exception CALL ptr[eax] DIV eax, 0 Malware Analysis and Instrumentation

  22. Instrumentation-based discovery H.A. Invalid control transfers Indirect control transfers Exception-based control transfers call 401000 Invalid Region jmp eax call ptr[eax] push eax ? ? ret xor eax, eax mov ebx, ptr[eax] Exception Handler Malware Analysis and Instrumentation

  23. Overwritten code discovery H.A. Dyninst write RWX RWX RWX 23 Malware Analysis and Instrumentation

  24. Overwritten code discovery H.A. Dyninst • Update after overwrite • Handle overwrite signal • instrument write loop exits • copy overwritten page • restore write permissions • resume execution • Update CFG when writes end • remove overwritten and unreachable blocks • parse at entry points to overwritten regions • remove write permissions • resume execution • Update after overwrite • Handle overwrite signal • instrument write loop exits • copy overwritten page • restore write permissions • resume execution • Update CFG when writes end • remove overwritten and unreachable blocks • parse at entry points to overwritten regions • remove write permissions • resume execution code write handler CFG update routine write cb cb R-X RWX R-X R-X 24 Malware Analysis and Instrumentation

  25. Overwritten code discovery H.A. Dyninst • Update after overwrite • Handle overwrite signal • instrument write loop exits • copy overwritten page • restore write permissions • resume execution • Update CFG when writes end • remove overwritten and unreachable blocks • parse at entry points to overwritten regions • remove write permissions • resume execution code write handler CFG update routine write cb cb R-X R-X RWX R-X 25 Malware Analysis and Instrumentation

  26. PC-sensitivity analysis S.R. SR-Dyninst process reloc_main: main: Relocate Analyze call ... data ... pop esi add esi, eax mov ebx, ptr[esi] ... push <orig> jmp 0 pop esi add esi, eax mov ebx, ptr[esi] ... Malware Analysis and Instrumentation

  27. Anti-anti patching S.R. code checksum routine patch xor eax, eax data add eax, ptr[ebx] add 4, ebx cmp ebx, 0x41000 jne .loop jmp 863828 add 4, ebx cmp ebx, 0x41000 jne .loop save state code patch emulate (add eax, ptr[ebx]) patch restore state cmp eax, .chksum jne .fail instrumentation pass fail fail shadow memory Malware Analysis and Instrumentation

  28. Address-space scanning S.R. code scan routine patch xor eax, eax data movptr[eax], ebx add 4, eax cmpeax, 0x0 jne .loop jmp 863828 add 4, eax cmpebx, 0x0 jne .loop save state patch code emulate (movptr[eax], ebx) patch restore state call chk_mem instrumentation pass fail fail segv_handler ... dyn_segv_handler ... ... Malware Analysis and Instrumentation

  29. The packers we’re studying Res. SR-Dyninst Packer Malware market share[1] Obfuscated Self-modifying Anti instru-mentation Dyninst √ UPX 9.45% √ PolyEnE 6.21% yes EXECryptor 4.06% yes yes yes x yes yes yes x Themida 2.95% yes yes yes PECompact 2.59% √ √ Upack 2.08% yes yes nPack 1.74% √ anti-debugging techniques √ Aspack 1.29% yes yes √ FSG 1.26% yes √ yes Nspack 0.89% yes yes Asprotect 0.43% yes yes √ x Armadillo 0.37% yes yes yes Yoda's Protector 0.33% yes yes yes √ √ WinUPack 0.17% yes yes MEW 0.13% √ yes Malware Analysis and Instrumentation [1] Packer (r)evolution. Panda Research, 2008. Two-month average Feb-March 2008.

  30. Improved Dyninst overhead Res. • Reduced relocation overhead despite emulation • Better handling of program features • Exceptions • Indirect control flow Malware Analysis and Instrumentation

  31. Conclusion • SR-Dyninst gives you • All the benefits of Dyninst on malware • Safer instrumentation on normal binaries • Ongoing work • Anti-debugger techniques • More descriptive CFGs • Automated defensive-mode activation • SR-Dyninst in next Dyninst release Malware Analysis and Instrumentation

More Related