Practical data analysis using ELK 2 - Stock

현재까지 구성된 ELK 스택을 이용해서 주식 분석을 실습한다.

http://blog.webkid.io/visualize-datasets-with-elk/

Collcet Datas

Datas site

https://finance.yahoo.com

Stock analysis Datas - Facebook

https://finance.yahoo.com/quote/FB/history?period1=1336316400&period2=1494082800&interval=1d&filter=history&frequency=1d

Get Ready-to-use Datas

Site에서 받은 데이터는 약간의 수정이 필요하다. 아래의 데이터는 바로 사용할 수 있는 데이터.

wget https://raw.githubusercontent.com/minsuk-heo/BigData/master/ch06/table.csv

Check ELASTICSEARCH & KIBANA are running

Check ELASTICSEARCH

service elasticsearch status

Check KIBANA

ps -ef | grep kibana

Config LOGSTASH

vi logstash_stock.conf

input {
  file {
    path => "/home/minsuk/Documents/git-repo/BigData/ch06/table.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"    
  }
}
filter {
  csv {
      separator => ","
      columns => ["Date","Open","High","Low","Close","Volume","Adj Close"]
  }
  mutate {convert => ["Open", "float"]}
  mutate {convert => ["High", "float"]}
  mutate {convert => ["Low", "float"]}
  mutate {convert => ["Close", "float"]}
}
output {  
    elasticsearch {
        hosts => "localhost"
        index => "stock"
    }
    stdout {}
}

• input -> file -> path
  • Edit Your own file path
  • e.g.) "/root/table.csv"

OR Download logstash_stock.conf file

wget https://raw.githubusercontent.com/minsuk-heo/BigData/master/ch06/logstash_stock.conf

Run LOGSTASH output to ELASTICSEARCH

/usr/share/logstash/bin/logstash -f ./logstash_stock.conf

Go KIBANA

http://localhost:5601/app/kibana#/management?_g=()

Add pattern

Visualize Tab

Line chart

Result

Dashboard Tab