1 / 31

Module 4 Database Tuning AND Data load

Module 4 Database Tuning AND Data load. Section 1 Database Tuning and Monitoring. Bulk Data Imports and Exports Using Data Pump. Previously, export (exp) and import (imp) utilities were used to import/export data from a DB

aneko
Download Presentation

Module 4 Database Tuning AND Data load

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Module 4 Database Tuning AND Data load ITEC 450 Section 1 Database Tuning and Monitoring

  2. Bulk Data Imports and Exports Using Data Pump • Previously, export (exp) and import (imp) utilities were used to import/export data from a DB • Data Pump export (expdp) and import (impdp) utilities are more versatile and much faster • Can be executed in parallel, failed/stopped jobs can be restarted, metadata can be filtered out, etc. • Exporting/importing is allowed at all logical layers: tables, schemas, groups of objects, or an entire DB • You export data and metadata • Database exports can used to migrate and upgrade between different versions of Oracle ITEC 450

  3. Bulk Data Imports and Exports Using Data Pump ITEC 450

  4. Exporting Using Data Pump • Steps: • Start a shell and • Type expdp help=y • Do a simple export expdp classmate/ classpass@oraclass DIRECTORY=dmpdir DUMPFILE=classmate.dmp • Create a directory object for Data Pump C/ CREATE OR REPLACE DIRECTORY DP AS 'C:\'; • Execute the expdp utility and export expdp system/password@oraclass DIRECTORY=dp DUMPFILE=system. dmp NOLOGFILE=Y • Export the CLASSMATE schema expdp system/password@oraclass DIRECTORY=dp DUMPFILE=classmate2. dmp SCHEMAS=('CLASSMATE') NOLOGFILE=Y ITEC 450

  5. Importing Using Data Pump • Data Pump is only available to DB- and OS level administrative personnel • Older utils. allow exports/imports across a network • Are secure because users have to have the schema password names to access schemas • Steps: • Start a shell • Type impdp help=y • The most likely use of these utilities are as fast backups and as an easy method of copying DBs • IMP and EXP are easier to use than Data Pump, with the same options, just fewer options ITEC 450

  6. Bulk Data Loads with SQL*Loader • SQL*Loader can perform magnificently in direct path, parallel mode, and using external tables • Direct path loads allow appending to tables • Some situations will cause single tables and even entire SQL*Loader executions to execute using a conventional path load • SQL*Loader is not limited to individual table loads • It can load into more than one table at once, considering all constraints • SQL*Loader can also perform fast direct loads with external tables ITEC 450

  7. Bulk Data Loads with SQL*Loader ITEC 450

  8. Direct Versus Conventional Path Loads • In a conventional path load rows are parsed into an array structure according to field specification, as defined by SQL Loader and table to be added to • A direct path is similar to a conventional one; but, load writes data to end of table into new block and extent structures, creating new blocks as it goes • Potentially much faster than conventional path loads • A parallel direct path load is a variation on a direct path load and is potentially faster than a direct one • The external table load creates and loads an external table ITEC 450

  9. SQL Loader Input Data Files • The input data files provide the data loaded in SQL Loader • Input data file contents can be fixed-length rows, variable-length rows, or a single stream (string) • Input data can consist of a single data file in control file, or a separate data set in the input data file • File does not have to exist and that input data can all be included in the control file • To avoid confusion, it is best to divide things into the different appropriate files ITEC 450

  10. The SQL Loader Control File • Control file contains a mapping between input data and table structures into which data is to be loaded LOAD DATA INFILE 'data\Chapter11\ch11.dat' INTO TABLE client APPEND FIELDS TERMINATED BY "," TRAILING NULLCOLS ( CLIENT_ID INTEGER, FIRST_NAME CHAR(10), LAST_NAME CHAR(20), CONTACT_PHONE CHAR(15), CONTACT_EMAIL CHAR(30) ) • The bad rows and discard rows can be defined too BADFILE 'solutions\Chapter11\bad.dat' DISCARDFILE 'solutions\Chapter11\discard.dat' ITEC 450

  11. Row Loading Options LOAD DATA INFILE ... BADFILE ... DISCARDFILE ... INTO TABLE table1 [ INSERT | REPLACE | TRUNCATE | APPEND ] FIELDS TERMINATED BY "," TRAILING NULLCOLS ( FIELD1 INTEGER EXTERNAL, FIELD2 INTEGER EXTERNAL, ... FIELDn ... ) • INSERT is the default ITEC 450

  12. Column Delimiters • Two examples: LOAD DATA INTO TABLE table1 TRUNCATE ( FIELD1POSITION(001:010) CHAR(10) TERMINATED BY WHITESPACE, FIELD2 POSITION(011:030) CHAR(20) TERMINATED BY WHITESPACE, ... , FIELDn ... ) LOAD DATA INTO TABLE table1 TRUNCATE FIELDS TERMINATED BY "," TRAILING NULLCOLS ( FIELD1 INTEGER EXTERNAL, FIELD2 INTEGER EXTERNAL, ... FIELDn ... ) ITEC 450

  13. The Parameter File • SQL*Loader can include a parameter file containing repeated settings, across multiple executions of SQL*Loader USERID = CLASSMATE/CLASSPASS@ORACLASS DISCARDMAX = 2 ERRORS = 1000000 • To load your new client rows: sqlldr control=<path>\data\Chapter11\ch11.ctl log=<path>\data\Chapter11\ch11.log parfile=<path>\data\Chapter11\ch11.par • Some of the most likely uses of SQL*Loader are to bulk load large amounts of data into a data warehouse, or when importing data from outside, into an existing database ITEC 450

  14. Examples of Performance Issues • A query has a response-time problem • A query used to run for 5 minutes, now for an hour and still running • A tablespace runs out of disk storage space • The batch window extends into day • Someone submitted a “query from hell” that just won’t stop running, and take away CPU and I/O resources ITEC 450

  15. Performance Influential Factors • Workload – a combination of online transactions, batch jobs, ad hoc queries, and other system activities outside of database • Throughput – the overall capability of the computer to process data, such as I/O speed • Resources – hardware and software tools, such as chips, memory, and database kernel • Optimization – query tuning, configuration change • Contention – two or more components of the workload are competing a single resource ITEC 450

  16. Database Performance Tuning Database performance tuning can be defined as the optimization of resource use to increase throughput and minimize contention, enabling the largest possible workload to be processed. • Identify the most troublesome areas • Application tuning – 80% of database performance problems are from poorly coded SQL or application logic • Database system tuning areas: • Memory allocation (buffer/cache for data) • Logging option (log cache, log size, log switch) • I/O efficiency (data contention, fragmentation) • Enabled features (parallelism, query rewrite) ITEC 450

  17. Performance Management Components • Monitoring – scanning the environment to collect performance information • Analysis – analyzing collected information to identify problem • Correction - modifying problematic behaviors ITEC 450

  18. Service-Level Management SLM is the disciplined methodology and procedures used to ensure adequate levels of services are delivered • A service level is a measure of operational behavior. • SLM focus on mainly availability, sometimes performance. • 99.9% uptime from 8:00 AM to 6:00 PM weekdays • Average response time for the transaction will be two seconds or less for workloads of 500 or fewer users • All involved parties, DBA and users, have to agree the service levels on stated objectives for availability and performance. ITEC 450

  19. DBMS Performance Basics • Do not over-tune – stop tuning when performance reaches a pre-defined service level • Remain focused – don’t tune multiple things at once • Do not panic – “I don’t know, but I’ll find out.” • Communicate clearly – keep users informed with progress, and coordinate discussions among all involved parties • Accept reality – Proactive is the goal, but reality is often that dealing with problems as they occur. ITEC 450

  20. Module 4 Database Tuning ITEC 450 Section 2 Managing and Monitoring Oracle Database

  21. Proactive Database Monitoring Alerts are used to monitor the database when particular metric thresholds are crossed, or can issue simply because an event has occurred. • For each alert, you can set critical and warning threshold • Default server-generated alerts: • CPU Time Per Call (warning at 8000 ms, critical at 10,000 ms) • Table space usage (warning at 85% full, critical at 97% full) • SQL Service Response Time • Recovery Area Space Usage ITEC 450

  22. Oracle Enterprise Manager ITEC 450

  23. Managing Alerts Using Oracle Enterprise Manager Warning and critical thresholds Response action, setting notification rules Read Chapter 10 in Oracle 2 Day DBA; and Chapter 5 in Oracle 2 Day DBA Tuning Guide Using DBMS_SERVER_ALERT Package Use DBMS_SERVER_ALERT.GET_THRESHOLD, SET_THRESHOLD SQL> connect sys/password as sysdba; SQL> descdbms_server_alert SQL> select reason from dba_outstanding_alerts; SQL> select reason, resolution from dba_alert_history; Metrics and alerts information – v$sysmetric, v$servicemetric, v$sysmetric_history Information about alert types – v$alert_types SQL> select distinct object_type from v$alert_types ITEC 450

  24. Alerts Section of OEM ITEC 450

  25. Automatic Workload Repository (AWR) AWR automatically collects and stores database performance statistics relating to problem detection and tuning. • A temporary in-memory collection of statistics in the SGA, accessible through dynamic views – v$sysstat, v$sessstat, v$session, v$session_wait, etc. • A persistent type of performance data, access by data dictionary views – dba_users, dba_source, dba_errors, etc. • By default, AWR generates snapshots once every hour, and retains the statistics for 8 days. • Read Chapter 3 in Oracle 2 Day DBA Tuning Guide ITEC 450

  26. Metrics Page from AWR ITEC 450

  27. AWR Reports Oracle provides a script generate summary reports: • Top five timed events • Wait events and latch activity • Time-model statistics • Buffer pool and PGA statistics and advisories SQL> @C:\app\Administrator\product\11.2.0\dbhome_1\RDBMS\ADMIN\awrrpt.sql; Type Specified: html Enter value for num_days: 1 Enter value for begin_snap: 84 Enter value for end_snap: 89 A report is generated under: C:\app\Administrator\product\11.2.0\dbhome_1\BIN. The file name can be specified or default. ITEC 450

  28. Active Session History (ASH) Oracle also provides statistics for all active sessions every second, instead of AWR with 60 minutes by default. • Current active session data – v$session, v$active_session_history SQL> select username, status, sid, serial#, program from v$session 2 where username is not null; SQL> select sample_time, session_id, event, module from v$active_session_history 2 where user_id = (select user_id from dba_users where username = 'SYSTEM'); • Read Chapter 4 in Oracle 2 Day DBA Tuning Guide ITEC 450

  29. Automatic Database Diagnostic Monitor ADDM identifies performance problems, and recommends actions, after the AWR takes a new snapshot. • ADDM diagnoses problems such as: • Expensive SQL statements • I/O performance issues • Resource bottlenecks, including memory and CPU bottlenecks • ADDM findings and recommendations • Finding types – problem, symptom, and informational • Recommendations – actions for each problem finding ITEC 450

  30. Managing ADDM Using Oracle Enterprise Manager Performance findings of the stored ADDM reports Using DBMS_ADVISOR Package CREATE_TASK – creates a new advisor task SET_DEFAULT_TASK – modifies default values of parameters within a task DELETE_TASK – deletes a specific task from the repository GET_TASK_REPORT – Displays the most recent ADDM report Using Oracle Provided Script SQL> @ C:\app\Administrator\product\11.2.0\dbhome_1\RDBMS\ADMIN \addmrpt.sql; A report is generated under: C:\app\Administrator\product\11.2.0\dbhome_1\BIN by default. The file name should start with addmrpt. ITEC 450

  31. Management Advisory Framework The advisors help in the performance tuning, identifying bottlenecks and suggesting optimal sizes for key database resources. • Memory advisor – recommendations regarding the optimal sizing of the SGA and PGA. • Tuning-related advisors • SQL tuning advisor – analyzing complex SQL statements and recommending way to improve performance • SQL access advisor – advising on creating new indexes, materialized views, or materialized view logs. • Space-Related advisors – segment advisor, undo advisor ITEC 450

More Related