1 / 13

brief introduction to relational database and big data analysis

brief introduction to relational database and big data analysis. Kunihiko Kaneko. Relational Database. Problems in data sharing Data is encoded in data files Other users can understand the data files ? Relational Database Relational Database is a standard of the followings

kochm
Download Presentation

brief introduction to relational database and big data analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. brief introduction torelational database andbig data analysis Kunihiko Kaneko

  2. Relational Database • Problems in data sharing • Data is encoded in data files • Other users can understand the data files ? • Relational Database Relational Database is a standard of the followings • data format (i.e. the way to encode data) • data operations (query and update) • the way to describe data format • the way to describe constraints

  3. describe data format relational database a relational database is a set of tables product(id, product_name, type, cost, created_at) data format description score(name, score, student_name, created_at, updated_at) data format description table_name(attribute name 1, attribute name 2, ...)

  4. describe constraints score(name, score, student_name, created_at, updated_at) data format description constraints description (SQL language) keywords: INTEGER, REAL, TEXT, DATETIME NOT NULL, UNIQUE, PRIMARY KEY, etc

  5. Data format of relational database relational database a relational database is a set of tables each table is a set of rows

  6. list of the table names in a database database command editor a table Database Browser (SQLiteman)

  7. description of data formats and constraints data sources various data formats relational database for data storage interactive command (written in SQL Language) programs (embedded SQL statements in a programming language)

  8. cat >/tmp/a.$$.sql <<-SQL create table quote ( seq INTEGER PRIMARY KEY NOT NULL, at datetime, USD real, GBP real, EUR real, CAD real, CHF real, SEK real, DKK real, NOK real, AUD real, NZD real, ZAR real, BHD real, IDR100 real, CNY real, HKD real, INR real, MYR real, PHP real, SGD real, KRW100 real, THB real, KWD real, SAR real, AED real, MXN real, PGK real, HUF real, CZK real, PLN real, RUB real, TRY real, a01 real, IDR100b real, CNYb real, MYRb real, KRW100b real, TWD real ); SQL cat /tmp/a.$$.sql | sqlite3 /tmp/quotedb cat >/tmp/a.$$.sql <<-SQL .mode csv .import /tmp/a.$$.csv quote SQL # tail -n +2 /tmp/Book1.csv > /tmp/a.$$.csv cat /tmp/a.$$.sql | sqlite3 /tmp/quotedb date a program to read the data source and store into database Currency exchange data data source description of data formats and constraints M <- table_to_melt(T, T$at, "%Y/%m/%d") # ggplot(M, aes(x=Date, y=Value, colour=factor(AttrNum))) + geom_point(size=1); Plot program

  9. Fukuoka-City map data A Digital elevation map data Plot Examples using Relational Database

  10. A Point Cloud data A Polygon data Three-dimensional Plot Examples using Relational Database

  11. Data Analysis Example – Future Prediction

  12. Data Analysis Example – Trend and Outlier

  13. Summary • Relational Database is easy • Describing data format and constraints is easy • Database browser (such as SQLiteman) • Relational Database can handle various type of data • Spatial • Temporal • There are already many types of data analysis methods

More Related