Stage/vuls
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update README (#387)

Browse Source 
Update Tutorial in README
This commit is contained in:
Kota Kanbe
2017-03-21 10:47:19 +09:00
committed by GitHub
parent 58afcfc49a
commit af5a1204bc
4 changed files with 268 additions and 119 deletions
Download Patch File Download Diff File Expand all files Collapse all files

187
README.ja.md
View File

@@ -68,7 +68,6 @@ Vulsのセットアップは以下の３パターンがある
- Dockerコンテナ上にセットアップ
see https://github.com/future-architect/vuls/tree/master/setup/docker
[日本語README](https://github.com/future-architect/vuls/blob/master/setup/docker/README.ja.md)
- Chefでセットアップ
see https://github.com/sadayuki-matsuno/vuls-cookbook
- 手動でセットアップ
@@ -76,13 +75,12 @@ Hello Vulsチュートリアルでは手動でのセットアップ方法で説
----
# Hello Vuls
# Tutorial: Local Scan Mode
本チュートリアルでは、Amazon EC2にVulsをセットアップし、自分に存在する脆弱性をスキャンする方法を説明する。
手順は以下の通り
1. Amazon Linuxを新規作成
1. 自分自身にSSH接続できるように設定
1. 必要なソフトウェアをインストール
1. go-cve-dictionaryをデプロイ
1. Vulsをデプロイ
@@ -105,21 +103,7 @@ Hello Vulsチュートリアルでは手動でのセットアップ方法で説
- [Q: How do I disable the automatic installation of critical and important security updates on initial launch?](https://aws.amazon.com/amazon-linux-ami/faqs/?nc1=h_ls)
## Step2. SSH setting
ローカルホストにSSH接続できるようにする。
SSHキーペアを作成し、公開鍵をauthorized_keysに追加する。
```bash
$ ssh-keygen -t rsa
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
$ chmod 600 ~/.ssh/authorized_keys
```
VulsはSSHパスワード認証をサポートしていない。SSH公開鍵鍵認証を使う必要がある。
また、パスワードありのSUDOもセキュリティ上の理由によりサポートしていないため、スキャン対象サーバに/etc/sudoersにNOPASSWDを設定して、パスワードなしでSUDO可能にする必要がある。
## Step3. Install requirements
## Step2. Install requirements
Vulsセットアップに必要な以下のソフトウェアをインストールする。
@@ -150,7 +134,7 @@ export PATH=$PATH:$GOROOT/bin:$GOPATH/bin
$ source /etc/profile.d/goenv.sh
```
## Step4. Deploy [go-cve-dictionary](https://github.com/kotakanbe/go-cve-dictionary)
## Step3. Deploy [go-cve-dictionary](https://github.com/kotakanbe/go-cve-dictionary)
```bash
$ sudo mkdir /var/log/vuls
@@ -177,7 +161,14 @@ $ ls -alh cve.sqlite3
-rw-r--r-- 1 ec2-user ec2-user 7.0M Mar 24 13:20 cve.sqlite3
```
## Step5. Deploy Vuls
日本語化したい場合は、JVNから脆弱性データベースを取得する。
```bash
$ cd $HOME
$ for i in `seq 1998 $(date +"%Y")`; do go-cve-dictionary fetchjvn -years $i; done
```
## Step4. Deploy Vuls
新規にターミナルを起動し、先ほど作成したEC2にSSH接続する。
```
@@ -188,44 +179,32 @@ $ cd vuls
$ make install
```
vulsを既にインストール済みでupdateしたい場合は
```bash
$ go get -u github.com/future-architect/vuls
```
で可能である。
go getでエラーが発生した場合は、以下の点を確認する。
- Gitのバージョンがv2以降か？
- Go依存パッケージの問題でgo getに失敗する場合は [deploying with glide](https://github.com/future-architect/vuls/blob/master/README.md#deploy-with-glide) を試す。
## Step6. Config
## Step5. Config
Vulsの設定ファイルを作成する（TOMLフォーマット）
設定ファイルのチェックを行う
```
$ cd $HOME
$ cat config.toml
[servers]
[servers.172-31-4-82]
host = "172.31.4.82"
port = "22"
user = "ec2-user"
keyPath = "/home/ec2-user/.ssh/id_rsa"
[servers.localhost]
host = "localhost"
port = "local"
```
## Step7. Check config.toml and settings on the server before scanning
Root権限が必要なディストリビューションもあるで、スキャン対象サーバの/etc/sudoersを変更する。
パスワードありのsudoはセキュリティ上の理由からサポート指定いないので、スキャンに必要なコマンドは、`NOPASSAWORD`として、remote host上の`etc/sudoers`に定義しておく。
See [Usage: Configtest#Check /etc/sudoers](#check-etcsudoers)
## Step6. Check config.toml and settings on the server before scanning
```
$ vuls configtest
```
詳細は [Usage: configtest](#usage-configtest) を参照
## Step8. Start Scanning
## Step7. Start Scanning
```
@@ -234,11 +213,11 @@ $ vuls scan
Scan Summary
============
172-31-4-82 amazon 2015.09 94 CVEs 103 updatable packages
localhost amazon 2015.09 94 CVEs 103 updatable packages
```
## Step9. Reporting
## Step8. Reporting
View one-line summary
@@ -247,7 +226,7 @@ $ vuls report -format-one-line-text -cvedb-path=$PWD/cve.sqlite3
One Line Summary
================
172-31-4-82 Total: 94 (High:19 Medium:54 Low:7 ?:14) 103 updatable packages
localhost Total: 94 (High:19 Medium:54 Low:7 ?:14) 103 updatable packages
```
@@ -256,7 +235,7 @@ View short summary.
```
$ vuls report -format-short-text -cvedb-path=$PWD/cve.sqlite3 --lang=ja
172-31-4-8 (amazon 2015.09)
localhost (amazon 2015.09)
===========================
Total: 94 (High:19 Medium:54 Low:7 ?:14) 103 updatable packages
@@ -276,7 +255,7 @@ View full report.
```
$ vuls report -format-full-text -cvedb-path=$PWD/cve.sqlite3 --lang=ja
172-31-4-82 (amazon 2015.09)
localhost (amazon 2015.09)
============================
Total: 94 (High:19 Medium:54 Low:7 ?:14) 103 updatable packages
@@ -307,7 +286,7 @@ Confidence 100 / YumUpdateSecurityMatch
... snip ...
```
## Step10. TUI
## Step9. TUI
Vulsにはスキャン結果の詳細を参照できるイカしたTUI(Terminal-Based User Interface)が付属している。
@@ -317,13 +296,105 @@ $ vuls tui
![Vuls-TUI](img/hello-vuls-tui.png)
## Step11. Web UI
## Step10. Web UI
[VulsRepo](https://github.com/usiusi360/vulsrepo)はスキャン結果をビボットテーブルのように分析可能にするWeb UIである。
[Online Demo](http://usiusi360.github.io/vulsrepo/)があるので試してみて。
----
# Tutorial: Remote Scan Mode
SSHを用いてリモートのホストをスキャンする方法を説明する。
1. Amazon Linuxを新規に1台作成（スキャン対象）
1. 必要なソフトウェアをインストール
1. RemoteホストにlocalhostからSSH可能にする
1. 設定
1. 設定ファイルと、スキャン対象サーバの設定のチェック
1. Scan
1. Reporting
先程のチュートリアルで作成したVulsサーバ(以下localhostと記述)を用いる。
## Step1. Launch Another Amazon Linux
[Tutorial: Local Scan Mode#Step1. Launch Amazon Linux](#step1-launch-amazon-linux)と同じ
新規にターミナルを開いて今作成したEC2にSSH接続する。
## Step2. Install Dependencies on the Remote Server
ディストリビューションによってはスキャンに必要な依存ソフトウェアをインストールする必要がある。
これらはリモートサーバ上に手動かAnsibleなどでインストールする。
依存ソフトウェアの詳細は [Dependencies on Target Servers](#dependencies-on-target-servers) を参照。
## Step3. Enable to SSH from Localhost
VulsはSSHパスワード認証をサポートしてない。SSHの鍵認証の設定をしなければならない。
localhost上でkeypairを作成し、remote host上のauthorized_keysに追加する。
- Localhost
```bash
$ ssh-keygen -t rsa
```
Copy ~/.ssh/id_rsa.pub to the clipboard.
- Remote Host
```
$ mkdir ~/.ssh
$ chmod 700 ~/.ssh
$ touch ~/.ssh/authorized_keys
$ chmod 600 ~/.ssh/authorized_keys
$ vim ~/.ssh/authorized_keys
```
Paste from the clipboard to ~/.ssh/.authorized_keys
パスワードありのsudoはセキュリティ上の理由からサポート指定いないので、スキャンに必要なコマンドは、`NOPASSAWORD`として、remote host上の`etc/sudoers`に定義しておく。
See [Usage: Configtest#Check /etc/sudoers](https://github.com/future-architect/vuls#check-etcsudoers)
## Step4. Config
- Localhost
```
$ cd $HOME
$ cat config.toml
[servers]
[servers.172-31-4-82]
host = "172.31.4.82"
port = "22"
user = "ec2-user"
keyPath = "/home/ec2-user/.ssh/id_rsa"
```
## Step5. Check config.toml and settings on the server before scanning
```
$ vuls configtest
```
see [Usage: configtest](#usage-configtest)
## Step6. Start Scanning
```
$ vuls scan
... snip ...
Scan Summary
============
172-31-4-82 amazon 2015.09 94 CVEs 103 updatable packages
```
## Step7. Reporting
See [Tutorial: Local Scan Mode#Step8. Reporting](#step8-reporting)
See [Tutorial: Local Scan Mode#Step9. TUI](#step9-tui)
See [Tutorial: Local Scan Mode#Step10. Web UI](#step10-web-ui)
----
# Architecture
## A. Scan via SSH Mode (Remote Scan Mode)
@@ -1242,7 +1313,7 @@ optional = [
```
$ vuls report \
-cvedb-type=mysql \
-cvedb-url="user:pass@tcp(localhost:3306)/dbname"
-cvedb-url="user:pass@tcp(localhost:3306)/dbname?parseTime=true"
```
----
@@ -1303,33 +1374,39 @@ VulsとDependency Checkを連携すると以下の利点がある
## Display the latest scan results
```
$ vuls tui -h
tui:
tui
[-cvedb-type=sqlite3|mysql]
[-cvedb-path=/path/to/cve.sqlite3]
[-cvedb-url=http://127.0.0.1:1323 or mysql connection string]
[-results-dir=/path/to/results]
[-refresh-cve]
[-results-dir=/path/to/results]
[-log-dir=/path/to/log]
[-debug]
[-debug-sql]
[-pipe]
-cvedb-path string
/path/to/sqlite3 (For get cve detail from cve.sqlite3)
/path/to/sqlite3 (For get cve detail from cve.sqlite3)
-cvedb-type string
DB type for fetching CVE dictionary (sqlite3 or mysql)
DB type for fetching CVE dictionary (sqlite3 or mysql) (default "sqlite3")
-cvedb-url string
http://cve-dictionary.com:8080 or mysql connection string
-debug
debug mode
-debug-sql
debug SQL
-log-dir string
/path/to/log (default "/var/log/vuls")
-pipe
Use stdin via PIPE
-refresh-cve
Refresh CVE information in JSON file under results dir
-results-dir string
/path/to/results
/path/to/results
```
Key binding is below.
| key | |

177
README.md
View File

@@ -78,17 +78,16 @@ You can run install and run Vuls on your machine with only a few commands.
see https://github.com/future-architect/vuls/tree/master/setup/docker
- Manually
Hello Vuls Tutorial shows how to setup vuls manually.
Tutorial shows how to setup vuls manually.
----
# Tutorial: Hello Vuls
# Tutorial: Local Scan Mode
This tutorial will let you scan the vulnerabilities on the localhost with Vuls.
This can be done in the following steps.
1. Launch Amazon Linux
1. Enable to ssh from localhost
1. Install requirements
1. Deploy go-cve-dictionary
1. Deploy Vuls
@@ -111,21 +110,7 @@ This can be done in the following steps.
- [Q: How do I disable the automatic installation of critical and important security updates on initial launch?](https://aws.amazon.com/amazon-linux-ami/faqs/?nc1=h_ls)
## Step2. SSH setting
This is required to ssh to itself.
Create a keypair then append public key to authorized_keys
```bash
$ ssh-keygen -t rsa
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
$ chmod 600 ~/.ssh/authorized_keys
```
Vuls doesn't support SSH password authentication. So you have to use SSH key-based authentication.
And also, SUDO with password is not supported for security reasons. So you have to define NOPASSWORD in /etc/sudoers on target servers.
## Step3. Install requirements
## Step2. Install requirements
Vuls requires the following packages.
@@ -156,7 +141,7 @@ Set the OS environment variable to current shell
$ source /etc/profile.d/goenv.sh
```
## Step4. Deploy [go-cve-dictionary](https://github.com/kotakanbe/go-cve-dictionary)
## Step3. Deploy [go-cve-dictionary](https://github.com/kotakanbe/go-cve-dictionary)
```bash
$ sudo mkdir /var/log/vuls
@@ -182,7 +167,7 @@ $ ls -alh cve.sqlite3
-rw-r--r-- 1 ec2-user ec2-user 7.0M Mar 24 13:20 cve.sqlite3
```
## Step5. Deploy Vuls
## Step4. Deploy Vuls
Launch a new terminal and SSH to the ec2 instance.
@@ -195,25 +180,24 @@ $ make install
```
The binary was built under `$GOPATH/bin`
## Step6. Config
## Step5. Config
Create a config file(TOML format).
Then check the config.
```
$ cd $HOME
$ cat config.toml
[servers]
[servers.172-31-4-82]
host = "172.31.4.82"
port = "22"
user = "ec2-user"
keyPath = "/home/ec2-user/.ssh/id_rsa"
[servers.localhost]
host = "localhost"
port = "local"
```
## Step7. Check config.toml and settings on the server before scanning
Root privilege is needed on Some distributions.
Sudo with password is not supported for security reasons. So you have to define NOPASSWORD in /etc/sudoers.
See [Usage: Configtest#Check /etc/sudoers](#check-etcsudoers)
## Step6. Check config.toml and settings on the server before scanning
```
$ vuls configtest
@@ -221,7 +205,7 @@ $ vuls configtest
see [Usage: configtest](#usage-configtest)
## Step8. Start Scanning
## Step7. Start Scanning
```
$ vuls scan
@@ -229,11 +213,11 @@ $ vuls scan
Scan Summary
============
172-31-4-82 amazon 2015.09 94 CVEs 103 updatable packages
localhost amazon 2015.09 94 CVEs 103 updatable packages
```
## Step9. Reporting
## Step8. Reporting
View one-line summary
@@ -242,7 +226,7 @@ $ vuls report -format-one-line-text -cvedb-path=$PWD/cve.sqlite3
One Line Summary
================
172-31-4-82 Total: 94 (High:19 Medium:54 Low:7 ?:14) 103 updatable packages
localhost Total: 94 (High:19 Medium:54 Low:7 ?:14) 103 updatable packages
```
@@ -251,7 +235,7 @@ View short summary.
```
$ vuls report -format-short-text
172-31-4-8 (amazon 2015.09)
localhost (amazon 2015.09)
===========================
Total: 94 (High:19 Medium:54 Low:7 ?:14) 103 updatable packages
@@ -273,7 +257,7 @@ View full report.
```
$ vuls report -format-full-text
172-31-4-82 (amazon 2015.09)
localhost (amazon 2015.09)
============================
Total: 94 (High:19 Medium:54 Low:7 ?:14) 103 updatable packages
@@ -300,7 +284,7 @@ Confidence 100 / YumUpdateSecurityMatch
... snip ...
```
## Step10. TUI
## Step9. TUI
Vuls has Terminal-Based User Interface to display the scan result.
@@ -310,13 +294,106 @@ $ vuls tui
![Vuls-TUI](img/hello-vuls-tui.png)
## Step11. Web UI
## Step10. Web UI
[VulsRepo](https://github.com/usiusi360/vulsrepo) is a awesome Web UI for Vuls.
Check it out the [Online Demo](http://usiusi360.github.io/vulsrepo/).
----
# Tutorial: Remote Scan Mode
This tutorial will let you scan the vulnerabilities on the remote host via SSH with Vuls.
This can be done in the following steps.
1. Launch Another Amazon Linux
1. Install Dependencies on the Remote Host
1. Enable to SSH from Localhost
1. Configuration
1. Check config.toml and settings on the server before scanning
1. Scan
1. Reporting
We will use the Vuls server (called localhost) created in the previous tutorial.
## Step1. Launch Another Amazon Linux
Same as [Tutorial: Local Scan Mode#Step1. Launch Amazon Linux](#step1-launch-amazon-linux)
Launch a new terminal and SSH to the Remote Server.
## Step2. Install Dependencies on the Remote Server
Depending on the distribution you need to install dependent modules.
Install these dependencies manually or using Ansible etc.
For details of dependent libraries, see [Dependencies on Target Servers](#dependencies-on-target-servers)
## Step3. Enable to SSH from Localhost
Vuls doesn't support SSH password authentication. So you have to use SSH key-based authentication.
Create a keypair on the localhost then append public key to authorized_keys on the remote host.
- Localhost
```bash
$ ssh-keygen -t rsa
```
Copy ~/.ssh/id_rsa.pub to the clipboard.
- Remote Host
```
$ mkdir ~/.ssh
$ chmod 700 ~/.ssh
$ touch ~/.ssh/authorized_keys
$ chmod 600 ~/.ssh/authorized_keys
$ vim ~/.ssh/authorized_keys
```
Paste from the clipboard to ~/.ssh/.authorized_keys
And also, SUDO with password is not supported for security reasons. So you have to define NOPASSWORD in /etc/sudoers on target servers.
See [Usage: Configtest#Check /etc/sudoers](https://github.com/future-architect/vuls#check-etcsudoers)
## Step4. Config
- Localhost
```
$ cd $HOME
$ cat config.toml
[servers]
[servers.172-31-4-82]
host = "172.31.4.82"
port = "22"
user = "ec2-user"
keyPath = "/home/ec2-user/.ssh/id_rsa"
```
## Step5. Check config.toml and settings on the server before scanning
```
$ vuls configtest
```
see [Usage: configtest](#usage-configtest)
## Step6. Start Scanning
```
$ vuls scan
... snip ...
Scan Summary
============
172-31-4-82 amazon 2015.09 94 CVEs 103 updatable packages
```
## Step7. Reporting
See [Tutorial: Local Scan Mode#Step8. Reporting](#step8-reporting)
See [Tutorial: Local Scan Mode#Step9. TUI](#step9-tui)
See [Tutorial: Local Scan Mode#Step10. Web UI](#step10-web-ui)
----
# Setup Vuls in a Docker Container
see https://github.com/future-architect/vuls/tree/master/setup/docker
@@ -343,7 +420,7 @@ On the aggregation server, you can refer to the scanning result of each scan tar
## Scanning Flow
![Vuls-Scan-Flow](img/vuls-scan-flow.png)
- Scan vulnerabilities on the servers via SSH and collect a list of the CVE ID
- To scan Docker containers, Vuls connect via ssh to the Docker host and then `docker exec` to the containers. So, no need to run sshd daemon on the containers.
- To scan Docker containers, Vuls connects via SSH to the Docker host and then `docker exec` to the containers. So, no need to run sshd daemon on the containers.
----
# Performance Considerations
@@ -617,7 +694,7 @@ configtest:
```
The configtest subcommand checks the following
- Whether vuls is able to connect via ssh to servers/containers defined in the config.toml
- Whether vuls is able to connect via SSH to servers/containers defined in the config.toml
- Whether Dependent package is installed on the scan target server
- Check /etc/sudoers
@@ -728,7 +805,7 @@ Vuls supports different types of SSH.
By Default, using a native Go implementation from crypto/ssh.
This is useful in situations where you may not have access to traditional UNIX tools.
To use external ssh command, specify this option.
To use external SSH command, specify this option.
This is useful If you want to use ProxyCommand or cipher algorithm of SSH that is not supported by native go implementation.
Don't forget to add below line to /etc/sudoers on the target servers. (username: vuls)
```
@@ -1239,7 +1316,7 @@ optional = [
```
$ vuls report \
-cvedb-type=mysql \
-cvedb-url="user:pass@tcp(localhost:3306)/dbname"
-cvedb-url="user:pass@tcp(localhost:3306)/dbname?parseTime=true"
```
----
@@ -1293,8 +1370,6 @@ How to integrate Vuls with OWASP Dependency Check
```
# Usage: TUI
## Display the latest scan results
@@ -1305,25 +1380,31 @@ tui:
[-cvedb-type=sqlite3|mysql]
[-cvedb-path=/path/to/cve.sqlite3]
[-cvedb-url=http://127.0.0.1:1323 or mysql connection string]
[-results-dir=/path/to/results]
[-refresh-cve]
[-results-dir=/path/to/results]
[-log-dir=/path/to/log]
[-debug]
[-debug-sql]
[-pipe]
-cvedb-path string
/path/to/sqlite3 (For get cve detail from cve.sqlite3) (default "/Users/kotakanbe/go/src/github.com/future-architect/vuls/cve.sqlite3")
/path/to/sqlite3 (For get cve detail from cve.sqlite3)
-cvedb-type string
DB type for fetching CVE dictionary (sqlite3 or mysql) (default "sqlite3")
-cvedb-url string
http://cve-dictionary.com:8080 or mysql connection string
-debug
debug mode
-debug-sql
debug SQL
-log-dir string
/path/to/log (default "/var/log/vuls")
-pipe
Use stdin via PIPE
-refresh-cve
Refresh CVE information in JSON file under results dir
-results-dir string
/path/to/results (default "/Users/kotakanbe/go/src/github.com/future-architect/vuls/results")
/path/to/results
```
Key binding is below.
@@ -1437,7 +1518,7 @@ Run with --debug, --sql-debug option.
- Adjusting Open File Limit
[Riak docs](http://docs.basho.com/riak/latest/ops/tuning/open-files-limit/) is awesome.
- Does Vuls accept ssh connections with fish-shell or old zsh as the login shell?
- Does Vuls accept SSH connections with fish-shell or old zsh as the login shell?
No, Vuls needs a user on the server for bash login. see also [#8](/../../issues/8)
- Windows

8
setup/docker/README.md
View File

@@ -85,7 +85,7 @@ vuls v0.1.xxx xxxx
1. fetch nvd (vuls/go-cve-dictionary)
1. configuration (vuls/vuls)
1. prepare (vuls/vuls)
1. configtest (vuls/vuls)
1. scan (vuls/vuls)
1. vulsrepo (vuls/vulsrepo)
@@ -113,7 +113,7 @@ port = "22"
user = "vuls-user"
keyPath = "/root/.ssh/id_rsa" # path to ssh private key in docker
```
　
```console
$ docker run --rm \
@@ -124,14 +124,14 @@ $ docker run --rm \
-config=./config.toml # path to config.toml in docker
```
## Step3. Prepare
## Step3. Configtest
```console
$ docker run --rm -it\
-v ~/.ssh:/root/.ssh:ro \
-v $PWD:/vuls \
-v $PWD/vuls-log:/var/log/vuls \
vuls/vuls prepare \
vuls/vuls configtest \
-config=./config.toml # path to config.toml in docker
```

15
setup/docker/vuls/latest/README.md
View File

@@ -39,7 +39,7 @@ Please see the [Documentation](https://github.com/future-architect/vuls)
$ docker run --rm vuls/vuls -v
```
## configtest
## config
Create config.toml referring to [this](https://github.com/future-architect/vuls#configuration).
@@ -52,25 +52,16 @@ port = "22"
user = "vuls-user"
keyPath = "/root/.ssh/id_rsa" # path to ssh private key in docker
```
　
```console
$ docker run --rm \
-v ~/.ssh:/root/.ssh:ro \
-v $PWD:/vuls \
-v $PWD/vuls-log:/var/log/vuls \
vuls/vuls configtest
```
## prepare
## configtest
```console
$ docker run --rm -it \
-v ~/.ssh:/root/.ssh:ro \
-v $PWD:/vuls \
-v $PWD/vuls-log:/var/log/vuls \
vuls/vuls prepare \
vuls/vuls configtest \
-config=./config.toml # path to config.toml in docker
```