Stage/vuls
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v0.5.0 (no backwards compatibility) (#478)

Browse Source 
* Change config.toml, Auto-generate UUIDs, change structure of optional field

* Detect processes affected by update using yum-ps (#482)

Detect processes affected by update using yum-ps

* Detect processes needs restart using checkrestart on Debian and Ubuntu.

* pass cpename by args when calling FillCveInfo (#513)

* fix new db (#502)

* Include Version,Revision in JSON

* Include hostname in JSON

* Update goval-dictionary's commit hash in Gopkg.lock

* Remove README.ja.md

* update packages (#596)

* fix: change ControlPath to .vuls of SSH option (#618)

* feat: checkrestart for Ubuntu and Debian (#622)

* feat: checkrestart for Ubuntu and Debian

* fix: dependencies check logic of configtest

* feat: need-restarting on RedHat

* refactor: Process.ProcName to Process.Name

* feat: detect a systemd service name of need-restarting-process

* feat: detect a systemd service name of need-restarting-process on Ubuntu

* feat: fill a service name of need-restarting-process, init-system

* Support NVD JSON and CVSS3 of JVN (#605)

* fix: compile errors

* fix: Show CVSS3 on TUI

* fix: test cases

* fix: Avoid null in JSON

* Fix maxCvssScore (#621)

* Fix maxCvssScore

* Update vulninfos.go

* fix(init): remove unnecessary log initialization

* refactor(nvd): use only json feed if exists json data. if not, use xml feed

* fix(scan): make Confidence slice

* feat(CWE): Display CWE name to TUI

* feat(cwe): import CWE defs in Japanese

* feat(cwe): add OWASP Top 10 ranking to CWE if applicable

* feat(scan): add -fast-root mode, implement scan/amazon.go

* refactor(const): change const name JVN to Jvn

* feat(scan): add -fast-root mode, implement scan/centos.go

* refactor(dep): update deps

* fix(amazon): deps check

* feat(scan): add -fast-root mode, implement scan/rhel.go

* feat(scan): add -fast-root mode, implement scan/oracle.go

* fix complile err

* feat(scan): add -fast-root mode, implement scan/debian.go

* fix testcase

* fix(amazon): scan using yum

* fix(configtest): change error message, status when no scannnable servers

* Fix(scan): detect init process logic

* fix(tui): display cvss as table format

* fix(scan): parse a output of reboot-notifier on CentOS6.9

* fix(tui): don't display score, vector when score is zero

* fix(scan): add -offline mode to suse scanner

* fix(scan): fix help message

* feat(scan): enable to define scan mode for each servers in config.toml #510

* refactor(config): chagne cpeNames to cpeURIs

* refactor(config): change dependencyCheckXMLPath to owaspDCXMLPath

* fix(config): containers -> containersIncluded, Excluded, containerType

* feature(report): enable to define cpeURIs for each contaner

* feature(report): enable to specify owasp dc xml path for each container

* fix(discover): fix a template displayed at the end of discover

* feature(report): add ignorePkgsRegexp #665

* feature(report): enable to define ignoreCves for each container #666

* fix(report): Displayed nothing in TUI detail area when CweID is nil

* Gopkg.toml diet

* feat(server): support server mode (#678)

* feat(server): support server mode

* Lock go version

* Use the latest kernel release among the installed release when the running kernel release is unknown

* Add TestViaHTTP

* Set logger to go-cve-dictionary client

* Add -to-localfile

* Add -to-http option to report

* Load -to-http conf from config.toml

* Support gost (#676)

* feat(gost): Support RedHat API

* feat(gost): Support Debian Security Tracker

* feat(db): display error msg when SQLite3 is locked at the beginning of reporting.

* feat(gost): TUI

* Only use RedHat information of installed packages

* feat(tui): show mitigation on TUI

* feat(gost): support redis backend

* fix test case

* fix nil pointer when db is nil

* fix(gost): detect vulns of src packages for Debian

* feat(gost): implement redis backend for gost redhat api

* feat(report): display fixState of unfixed pkgs

* fix(report): display distincted cweIDs

* feat(slack): display gost info

* feat(slack): display mitigation

* feat(report): display available patch state as fixed/total

* fix(tui): display - if source of reference is empty

* update deps

* fix(report): key in ScanResult JSON be lowerCamelcase.

* some keys to lower camel

* fix(configtest): dep check logic of yum-plugin-ps

* fix(tui): format

* feat(report): add -format-list option

* fix(report): -format-full-text

* fix(report): report -format-full-text

* fix(report): display v3 score detected by gost

* fix(scan): scan in fast mode if not defined in config.toml

* fix(gost): fetch RedHat data for fixed CVEs

* feat(report): show number of cves detected in each database

* fix(report): show new version as `Unknown` in offline and fast scan mode

* fix(report): fix num of upadtable and fixed

* fix(report): set `Not fixed yet` if packageStatus is empty

* refact(gost): make convertToModel public

* fix(test): fix test case

* update deps

* fix(report): include gost score in MaxCvssScore

* [WIP] feat(config): enable to set options in config.toml instead of cmd opt (#690)

* feat(config): enable to set options in config.toml instead of cmd opt

* fix(config): change Conf.Report.Slack to Conf.Slack

* fix(discover): change tempalte

* fix(report): fix config.toml auto-generate with -uuid

* Add endpoint for health check and change endpoint

* refact(cmd): refactor flag set

* fix(report): enable to specify opts with cmd arg and env value

* fix(scan): enable to parse the release version of amazon linux 2

* add(report) add -to-saas option (#695)

* add(report) add -to-saas option

* ignore other writer if -to-saas

* fix(saas) fix bug

* fix(scan): need-restarting needs internet connection

* fix(scan,configtest): check scan mode

* refactor(scan): change func name

* fix(suse): support offline mode, bug fix on AWS, zypper --no-color

* fix(tui): fix nil pointer when no vulns in tui

* feat(report): enable to define CPE FS format in config.toml

* fix(vet): fix warnings of go vet

* fix(travis): go version to 1.11

* update deps
This commit is contained in:
Kota Kanbe
2018-08-27 13:51:09 +09:00
committed by GitHub
parent d785fc2a54
commit 44fa2c5800
82 changed files with 14019 additions and 2485 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
report/azureblob.go
View File

@@ -67,9 +67,9 @@ func (w AzureBlobWriter) Write(rs ...models.ScanResult) (err error) {
}
}
if c.Conf.FormatShortText {
if c.Conf.FormatList {
k := key + "_short.txt"
b := []byte(formatShortPlainText(r))
b := []byte(formatList(r))
if err := createBlockBlob(cli, k, b); err != nil {
return err
}
@@ -111,19 +111,19 @@ func CheckIfAzureContainerExists() error {
found := false
for _, con := range r.Containers {
if con.Name == c.Conf.AzureContainer {
if con.Name == c.Conf.Azure.ContainerName {
found = true
break
}
}
if !found {
return fmt.Errorf("Container not found. Container: %s", c.Conf.AzureContainer)
return fmt.Errorf("Container not found. Container: %s", c.Conf.Azure.ContainerName)
}
return nil
}
func getBlobClient() (storage.BlobStorageClient, error) {
api, err := storage.NewBasicClient(c.Conf.AzureAccount, c.Conf.AzureKey)
api, err := storage.NewBasicClient(c.Conf.Azure.AccountName, c.Conf.Azure.AccountKey)
if err != nil {
return storage.BlobStorageClient{}, err
}
@@ -139,11 +139,11 @@ func createBlockBlob(cli storage.BlobStorageClient, k string, b []byte) error {
k = k + ".gz"
}
ref := cli.GetContainerReference(c.Conf.AzureContainer)
ref := cli.GetContainerReference(c.Conf.Azure.ContainerName)
blob := ref.GetBlobReference(k)
if err := blob.CreateBlockBlobFromReader(bytes.NewReader(b), nil); err != nil {
return fmt.Errorf("Failed to upload data to %s/%s, %s",
c.Conf.AzureContainer, k, err)
c.Conf.Azure.ContainerName, k, err)
}
return nil
}

107
report/cve_client.go
View File

@@ -28,10 +28,8 @@ import (
"github.com/future-architect/vuls/config"
"github.com/future-architect/vuls/util"
cveconfig "github.com/kotakanbe/go-cve-dictionary/config"
cvedb "github.com/kotakanbe/go-cve-dictionary/db"
cve "github.com/kotakanbe/go-cve-dictionary/models"
log "github.com/sirupsen/logrus"
)
// CveClient is api client of CVE disctionary service.
@@ -43,12 +41,12 @@ type cvedictClient struct {
}
func (api *cvedictClient) initialize() {
api.baseURL = config.Conf.CveDBURL
api.baseURL = config.Conf.CveDict.URL
}
func (api cvedictClient) CheckHealth() error {
if !api.isFetchViaHTTP() {
util.Log.Debugf("get cve-dictionary from %s", config.Conf.CveDBType)
util.Log.Debugf("get cve-dictionary from %s", config.Conf.CveDict.Type)
return nil
}
@@ -70,12 +68,25 @@ type response struct {
CveDetail cve.CveDetail
}
func (api cvedictClient) FetchCveDetails(cveIDs []string) (cveDetails []*cve.CveDetail, err error) {
func (api cvedictClient) FetchCveDetails(driver cvedb.DB, cveIDs []string) (cveDetails []cve.CveDetail, err error) {
if !api.isFetchViaHTTP() {
return api.FetchCveDetailsFromCveDB(cveIDs)
for _, cveID := range cveIDs {
cveDetail, err := driver.Get(cveID)
if err != nil {
return nil, fmt.Errorf("Failed to fetch CVE. err: %s", err)
}
if len(cveDetail.CveID) == 0 {
cveDetails = append(cveDetails, cve.CveDetail{
CveID: cveID,
})
} else {
cveDetails = append(cveDetails, *cveDetail)
}
}
return
}
api.baseURL = config.Conf.CveDBURL
api.baseURL = config.Conf.CveDict.URL
reqChan := make(chan string, len(cveIDs))
resChan := make(chan response, len(cveIDs))
errChan := make(chan error, len(cveIDs))
@@ -112,55 +123,25 @@ func (api cvedictClient) FetchCveDetails(cveIDs []string) (cveDetails []*cve.Cve
select {
case res := <-resChan:
if len(res.CveDetail.CveID) == 0 {
cveDetails = append(cveDetails, &cve.CveDetail{
cveDetails = append(cveDetails, cve.CveDetail{
CveID: res.Key,
})
} else {
cveDetails = append(cveDetails, &res.CveDetail)
cveDetails = append(cveDetails, res.CveDetail)
}
case err := <-errChan:
errs = append(errs, err)
case <-timeout:
return []*cve.CveDetail{}, fmt.Errorf("Timeout Fetching CVE")
return nil, fmt.Errorf("Timeout Fetching CVE")
}
}
if len(errs) != 0 {
return []*cve.CveDetail{},
return nil,
fmt.Errorf("Failed to fetch CVE. err: %v", errs)
}
return
}
func (api cvedictClient) FetchCveDetailsFromCveDB(cveIDs []string) (cveDetails []*cve.CveDetail, err error) {
util.Log.Debugf("open cve-dictionary db (%s)", config.Conf.CveDBType)
cveconfig.Conf.DBType = config.Conf.CveDBType
if config.Conf.CveDBType == "sqlite3" {
cveconfig.Conf.DBPath = config.Conf.CveDBPath
} else {
cveconfig.Conf.DBPath = config.Conf.CveDBURL
}
cveconfig.Conf.DebugSQL = config.Conf.DebugSQL
var driver cvedb.DB
if driver, err = cvedb.NewDB(cveconfig.Conf.DBType, cveconfig.Conf.DBPath, cveconfig.Conf.DebugSQL); err != nil {
log.Error(err)
return []*cve.CveDetail{}, fmt.Errorf("Failed to New DB. err: %s", err)
}
defer driver.CloseDB()
for _, cveID := range cveIDs {
cveDetail := driver.Get(cveID)
if len(cveDetail.CveID) == 0 {
cveDetails = append(cveDetails, &cve.CveDetail{
CveID: cveID,
})
} else {
cveDetails = append(cveDetails, cveDetail)
}
}
return
}
func (api cvedictClient) httpGet(key, url string, resChan chan<- response, errChan chan<- error) {
var body string
var errs []error
@@ -195,36 +176,30 @@ func (api cvedictClient) httpGet(key, url string, resChan chan<- response, errCh
}
}
type responseGetCveDetailByCpeName struct {
CpeName string
CveDetails []cve.CveDetail
}
func (api cvedictClient) isFetchViaHTTP() bool {
// Default value of CveDBType is sqlite3
if config.Conf.CveDBURL != "" && config.Conf.CveDBType == "sqlite3" {
if config.Conf.CveDict.URL != "" && config.Conf.CveDict.Type == "sqlite3" {
return true
}
return false
}
func (api cvedictClient) FetchCveDetailsByCpeName(cpeName string) ([]*cve.CveDetail, error) {
func (api cvedictClient) FetchCveDetailsByCpeName(driver cvedb.DB, cpeName string) ([]cve.CveDetail, error) {
if api.isFetchViaHTTP() {
api.baseURL = config.Conf.CveDBURL
api.baseURL = config.Conf.CveDict.URL
url, err := util.URLPathJoin(api.baseURL, "cpes")
if err != nil {
return []*cve.CveDetail{}, err
return nil, err
}
query := map[string]string{"name": cpeName}
util.Log.Debugf("HTTP Request to %s, query: %#v", url, query)
return api.httpPost(cpeName, url, query)
}
return api.FetchCveDetailsByCpeNameFromDB(cpeName)
return driver.GetByCpeURI(cpeName)
}
func (api cvedictClient) httpPost(key, url string, query map[string]string) ([]*cve.CveDetail, error) {
func (api cvedictClient) httpPost(key, url string, query map[string]string) ([]cve.CveDetail, error) {
var body string
var errs []error
var resp *http.Response
@@ -245,33 +220,13 @@ func (api cvedictClient) httpPost(key, url string, query map[string]string) ([]*
}
err := backoff.RetryNotify(f, backoff.NewExponentialBackOff(), notify)
if err != nil {
return []*cve.CveDetail{}, fmt.Errorf("HTTP Error %s", err)
return nil, fmt.Errorf("HTTP Error %s", err)
}
cveDetails := []*cve.CveDetail{}
cveDetails := []cve.CveDetail{}
if err := json.Unmarshal([]byte(body), &cveDetails); err != nil {
return []*cve.CveDetail{},
return nil,
fmt.Errorf("Failed to Unmarshall. body: %s, err: %s", body, err)
}
return cveDetails, nil
}
func (api cvedictClient) FetchCveDetailsByCpeNameFromDB(cpeName string) (cveDetails []*cve.CveDetail, err error) {
util.Log.Debugf("open cve-dictionary db (%s)", config.Conf.CveDBType)
cveconfig.Conf.DBType = config.Conf.CveDBType
if config.Conf.CveDBType == "sqlite3" {
cveconfig.Conf.DBPath = config.Conf.CveDBPath
} else {
cveconfig.Conf.DBPath = config.Conf.CveDBURL
}
cveconfig.Conf.DebugSQL = config.Conf.DebugSQL
var driver cvedb.DB
if driver, err = cvedb.NewDB(cveconfig.Conf.DBType, cveconfig.Conf.DBPath, cveconfig.Conf.DebugSQL); err != nil {
log.Error(err)
return []*cve.CveDetail{}, fmt.Errorf("Failed to New DB. err: %s", err)
}
util.Log.Debugf("Opening DB (%s).", driver.Name())
return driver.GetByCpeName(cpeName), nil
}

158
report/db_client.go Normal file
View File

@@ -0,0 +1,158 @@
package report
import (
"fmt"
"os"
"github.com/future-architect/vuls/config"
"github.com/future-architect/vuls/util"
gostdb "github.com/knqyf263/gost/db"
cvedb "github.com/kotakanbe/go-cve-dictionary/db"
ovaldb "github.com/kotakanbe/goval-dictionary/db"
)
// DBClient is a dictionarie's db client for reporting
type DBClient struct {
CveDB cvedb.DB
OvalDB ovaldb.DB
GostDB gostdb.DB
}
// DBClientConf has a configuration of Vulnerability DBs
type DBClientConf struct {
CveDictCnf config.GoCveDictConf
OvalDictCnf config.GovalDictConf
GostCnf config.GostConf
DebugSQL bool
}
func (c DBClientConf) isCveDBViaHTTP() bool {
return c.CveDictCnf.URL != "" && c.CveDictCnf.Type == "sqlite3"
}
func (c DBClientConf) isOvalViaHTTP() bool {
return c.OvalDictCnf.URL != "" && c.OvalDictCnf.Type == "sqlite3"
}
func (c DBClientConf) isGostViaHTTP() bool {
return c.GostCnf.URL != "" && c.GostCnf.Type == "sqlite3"
}
// NewDBClient returns db clients
func NewDBClient(cnf DBClientConf) (dbclient *DBClient, locked bool, err error) {
cveDriver, locked, err := NewCveDB(cnf)
if err != nil {
return nil, locked, err
}
ovaldb, locked, err := NewOvalDB(cnf)
if locked {
return nil, true, fmt.Errorf("OvalDB is locked: %s",
cnf.OvalDictCnf.SQLite3Path)
} else if err != nil {
util.Log.Warnf("Unable to use OvalDB: %s, err: %s",
cnf.OvalDictCnf.SQLite3Path, err)
}
gostdb, locked, err := NewGostDB(cnf)
if locked {
return nil, true, fmt.Errorf("gostDB is locked: %s",
cnf.GostCnf.SQLite3Path)
} else if err != nil {
util.Log.Warnf("Unable to use gostDB: %s, err: %s",
cnf.GostCnf.SQLite3Path, err)
}
return &DBClient{
CveDB: cveDriver,
OvalDB: ovaldb,
GostDB: gostdb,
}, false, nil
}
// NewCveDB returns cve db client
func NewCveDB(cnf DBClientConf) (driver cvedb.DB, locked bool, err error) {
if cnf.isCveDBViaHTTP() {
return nil, false, nil
}
util.Log.Debugf("open cve-dictionary db (%s)", cnf.CveDictCnf.Type)
path := cnf.CveDictCnf.URL
if cnf.CveDictCnf.Type == "sqlite3" {
path = cnf.CveDictCnf.SQLite3Path
}
util.Log.Debugf("Open cve-dictionary db (%s): %s", cnf.CveDictCnf.Type, path)
driver, locked, err = cvedb.NewDB(cnf.CveDictCnf.Type, path, cnf.DebugSQL)
if err != nil {
err = fmt.Errorf("Failed to init CVE DB. err: %s, path: %s", err, path)
return nil, locked, err
}
return driver, false, nil
}
// NewOvalDB returns oval db client
func NewOvalDB(cnf DBClientConf) (driver ovaldb.DB, locked bool, err error) {
if cnf.isOvalViaHTTP() {
return nil, false, nil
}
path := cnf.OvalDictCnf.URL
if cnf.OvalDictCnf.Type == "sqlite3" {
path = cnf.OvalDictCnf.SQLite3Path
if _, err := os.Stat(path); os.IsNotExist(err) {
util.Log.Warnf("--ovaldb-path=%s is not found. It's recommended to use OVAL to improve scanning accuracy. For details, see https://github.com/kotakanbe/goval-dictionary#usage", path)
return nil, false, nil
}
}
util.Log.Debugf("Open oval-dictionary db (%s): %s", cnf.OvalDictCnf.Type, path)
driver, locked, err = ovaldb.NewDB("", cnf.OvalDictCnf.Type, path, cnf.DebugSQL)
if err != nil {
err = fmt.Errorf("Failed to new OVAL DB. err: %s", err)
if locked {
return nil, true, err
}
return nil, false, err
}
return driver, false, nil
}
// NewGostDB returns db client for Gost
func NewGostDB(cnf DBClientConf) (driver gostdb.DB, locked bool, err error) {
if cnf.isGostViaHTTP() {
return nil, false, nil
}
path := cnf.GostCnf.URL
if cnf.GostCnf.Type == "sqlite3" {
path = cnf.GostCnf.SQLite3Path
if _, err := os.Stat(path); os.IsNotExist(err) {
util.Log.Warnf("--gostdb-path=%s is not found. If the scan target server is Debian, RHEL or CentOS, it's recommended to use gost to improve scanning accuracy. To use gost database, see https://github.com/knqyf263/gost#fetch-redhat", path)
return nil, false, nil
}
}
util.Log.Debugf("Open gost db (%s): %s", cnf.GostCnf.Type, path)
if driver, locked, err = gostdb.NewDB(cnf.GostCnf.Type, path, cnf.DebugSQL); err != nil {
if locked {
util.Log.Errorf("gostDB is locked: %s", err)
return nil, true, err
}
return nil, false, err
}
return driver, false, nil
}
// CloseDB close dbs
func (d DBClient) CloseDB() {
if d.CveDB != nil {
if err := d.CveDB.CloseDB(); err != nil {
util.Log.Errorf("Failed to close DB: %s", err)
}
}
if d.OvalDB != nil {
if err := d.OvalDB.CloseDB(); err != nil {
util.Log.Errorf("Failed to close DB: %s", err)
}
}
}

62
report/http.go Normal file
View File

@@ -0,0 +1,62 @@
/* Vuls - Vulnerability Scanner
Copyright (C) 2016 Future Architect, Inc. Japan.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package report
import (
"bytes"
"encoding/json"
"net/http"
"github.com/pkg/errors"
c "github.com/future-architect/vuls/config"
"github.com/future-architect/vuls/models"
)
// HTTPRequestWriter writes results to HTTP request
type HTTPRequestWriter struct{}
// Write sends results as HTTP response
func (w HTTPRequestWriter) Write(rs ...models.ScanResult) (err error) {
for _, r := range rs {
b := new(bytes.Buffer)
json.NewEncoder(b).Encode(r)
_, err = http.Post(c.Conf.HTTP.URL, "application/json; charset=utf-8", b)
if err != nil {
return err
}
}
return nil
}
// HTTPResponseWriter writes results to HTTP response
type HTTPResponseWriter struct {
Writer http.ResponseWriter
}
// Write sends results as HTTP response
func (w HTTPResponseWriter) Write(rs ...models.ScanResult) (err error) {
res, err := json.Marshal(rs)
if err != nil {
return errors.Wrap(err, "Failed to marshal scah results")
}
w.Writer.Header().Set("Content-Type", "application/json")
_, err = w.Writer.Write(res)
return err
}

12
report/localfile.go
View File

@@ -72,7 +72,7 @@ func (w LocalFileWriter) Write(rs ...models.ScanResult) (err error) {
}
}
if c.Conf.FormatShortText {
if c.Conf.FormatList {
var p string
if c.Conf.Diff {
p = path + "_short_diff.txt"
@@ -81,7 +81,7 @@ func (w LocalFileWriter) Write(rs ...models.ScanResult) (err error) {
}
if err := writeFile(
p, []byte(formatShortPlainText(r)), 0600); err != nil {
p, []byte(formatList(r)), 0600); err != nil {
return fmt.Errorf(
"Failed to write text files. path: %s, err: %s", p, err)
}
@@ -131,11 +131,5 @@ func writeFile(path string, data []byte, perm os.FileMode) error {
}
path = path + ".gz"
}
if err := ioutil.WriteFile(
path, []byte(data), perm); err != nil {
return err
}
return nil
return ioutil.WriteFile(path, []byte(data), perm)
}

495
report/report.go
View File

@@ -18,13 +18,28 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
package report
import (
"bytes"
"fmt"
"io/ioutil"
"os"
"reflect"
"regexp"
"sort"
"strings"
"time"
"github.com/BurntSushi/toml"
c "github.com/future-architect/vuls/config"
"github.com/future-architect/vuls/contrib/owasp-dependency-check/parser"
"github.com/future-architect/vuls/cwe"
"github.com/future-architect/vuls/gost"
"github.com/future-architect/vuls/models"
"github.com/future-architect/vuls/oval"
"github.com/future-architect/vuls/util"
"github.com/hashicorp/uuid"
gostdb "github.com/knqyf263/gost/db"
cvedb "github.com/kotakanbe/go-cve-dictionary/db"
ovaldb "github.com/kotakanbe/goval-dictionary/db"
)
const (
@@ -33,16 +48,49 @@ const (
)
// FillCveInfos fills CVE Detailed Information
func FillCveInfos(rs []models.ScanResult, dir string) ([]models.ScanResult, error) {
func FillCveInfos(dbclient DBClient, rs []models.ScanResult, dir string) ([]models.ScanResult, error) {
var filledResults []models.ScanResult
reportedAt := time.Now()
hostname, _ := os.Hostname()
for _, r := range rs {
if c.Conf.RefreshCve || needToRefreshCve(r) {
if err := FillCveInfo(&r); err != nil {
cpeURIs := []string{}
if len(r.Container.ContainerID) == 0 {
cpeURIs = c.Conf.Servers[r.ServerName].CpeNames
owaspDCXMLPath := c.Conf.Servers[r.ServerName].OwaspDCXMLPath
if owaspDCXMLPath != "" {
cpes, err := parser.Parse(owaspDCXMLPath)
if err != nil {
return nil, fmt.Errorf("Failed to read OWASP Dependency Check XML: %s, %s, %s",
r.ServerName, owaspDCXMLPath, err)
}
cpeURIs = append(cpeURIs, cpes...)
}
} else {
if s, ok := c.Conf.Servers[r.ServerName]; ok {
if con, ok := s.Containers[r.Container.Name]; ok {
cpeURIs = con.Cpes
owaspDCXMLPath := con.OwaspDCXMLPath
if owaspDCXMLPath != "" {
cpes, err := parser.Parse(owaspDCXMLPath)
if err != nil {
return nil, fmt.Errorf("Failed to read OWASP Dependency Check XML: %s, %s, %s",
r.ServerInfo(), owaspDCXMLPath, err)
}
cpeURIs = append(cpeURIs, cpes...)
}
}
}
}
if err := FillCveInfo(dbclient, &r, cpeURIs); err != nil {
return nil, err
}
r.Lang = c.Conf.Lang
r.ReportedAt = reportedAt
r.ReportedVersion = c.Version
r.ReportedRevision = c.Revision
r.ReportedBy = hostname
r.Config.Report = c.Conf
r.Config.Report.Servers = map[string]c.ServerInfo{
r.ServerName: c.Conf.Servers[r.ServerName],
@@ -69,7 +117,7 @@ func FillCveInfos(rs []models.ScanResult, dir string) ([]models.ScanResult, erro
}
filledResults = []models.ScanResult{}
for _, r := range diff {
if err := fillCveDetail(&r); err != nil {
if err := fillCveDetail(dbclient.CveDB, &r); err != nil {
return nil, err
}
filledResults = append(filledResults, r)
@@ -79,8 +127,9 @@ func FillCveInfos(rs []models.ScanResult, dir string) ([]models.ScanResult, erro
filtered := []models.ScanResult{}
for _, r := range filledResults {
r = r.FilterByCvssOver(c.Conf.CvssScoreOver)
r = r.FilterIgnoreCves(c.Conf.Servers[r.ServerName].IgnoreCves)
r = r.FilterIgnoreCves()
r = r.FilterUnfixed()
r = r.FilterIgnorePkgs()
if c.Conf.IgnoreUnscoredCves {
r.ScannedCves = r.ScannedCves.FindScoredVulns()
}
@@ -90,48 +139,73 @@ func FillCveInfos(rs []models.ScanResult, dir string) ([]models.ScanResult, erro
}
// FillCveInfo fill scanResult with cve info.
func FillCveInfo(r *models.ScanResult) error {
func FillCveInfo(dbclient DBClient, r *models.ScanResult, cpeURIs []string) error {
util.Log.Debugf("need to refresh")
util.Log.Infof("Fill CVE detailed information with OVAL")
if err := FillWithOval(r); err != nil {
return fmt.Errorf("Failed to fill OVAL information: %s", err)
nCVEs, err := FillWithOval(dbclient.OvalDB, r)
if err != nil {
return fmt.Errorf("Failed to fill with OVAL: %s", err)
}
util.Log.Infof("%s: %d CVEs are detected with OVAL",
r.FormatServerName(), nCVEs)
for i, v := range r.ScannedCves {
for j, p := range v.AffectedPackages {
if p.NotFixedYet && p.FixState == "" {
p.FixState = "Not fixed yet"
r.ScannedCves[i].AffectedPackages[j] = p
}
}
}
nCVEs, err = fillVulnByCpeURIs(dbclient.CveDB, r, cpeURIs)
if err != nil {
return fmt.Errorf("Failed to detect vulns of %s: %s", cpeURIs, err)
}
util.Log.Infof("%s: %d CVEs are detected with CPE", r.FormatServerName(), nCVEs)
nCVEs, err = FillWithGost(dbclient.GostDB, r)
if err != nil {
return fmt.Errorf("Failed to fill with gost: %s", err)
}
util.Log.Infof("%s: %d unfixed CVEs are detected with gost",
r.FormatServerName(), nCVEs)
util.Log.Infof("Fill CVE detailed information with CVE-DB")
if err := fillWithCveDB(r); err != nil {
return fmt.Errorf("Failed to fill CVE information: %s", err)
if err := fillCveDetail(dbclient.CveDB, r); err != nil {
return fmt.Errorf("Failed to fill with CVE: %s", err)
}
for cveID := range r.ScannedCves {
vinfo := r.ScannedCves[cveID]
r.ScannedCves[cveID] = *vinfo.NilToEmpty()
}
fillCweDict(r)
return nil
}
// fillCveDetail fetches NVD, JVN from CVE Database, and then set to fields.
func fillCveDetail(r *models.ScanResult) error {
// fillCveDetail fetches NVD, JVN from CVE Database
func fillCveDetail(driver cvedb.DB, r *models.ScanResult) error {
var cveIDs []string
for _, v := range r.ScannedCves {
cveIDs = append(cveIDs, v.CveID)
}
ds, err := CveClient.FetchCveDetails(cveIDs)
ds, err := CveClient.FetchCveDetails(driver, cveIDs)
if err != nil {
return err
}
for _, d := range ds {
nvd := models.ConvertNvdToModel(d.CveID, d.Nvd)
nvd := models.ConvertNvdJSONToModel(d.CveID, d.NvdJSON)
if nvd == nil {
nvd = models.ConvertNvdXMLToModel(d.CveID, d.NvdXML)
}
jvn := models.ConvertJvnToModel(d.CveID, d.Jvn)
for cveID, vinfo := range r.ScannedCves {
if vinfo.CveID == d.CveID {
if vinfo.CveContents == nil {
vinfo.CveContents = models.CveContents{}
}
for _, con := range []models.CveContent{*nvd, *jvn} {
if !con.Empty() {
vinfo.CveContents[con.Type] = con
for _, con := range []*models.CveContent{nvd, jvn} {
if con != nil && !con.Empty() {
vinfo.CveContents[con.Type] = *con
}
}
r.ScannedCves[cveID] = vinfo
@@ -142,23 +216,11 @@ func fillCveDetail(r *models.ScanResult) error {
return nil
}
func fillWithCveDB(r *models.ScanResult) error {
sInfo := c.Conf.Servers[r.ServerName]
if err := fillVulnByCpeNames(sInfo.CpeNames, r.ScannedCves); err != nil {
return err
}
if err := fillCveDetail(r); err != nil {
return err
}
return nil
}
// FillWithOval fetches OVAL database, and then set to fields.
func FillWithOval(r *models.ScanResult) (err error) {
// FillWithOval fetches OVAL database
func FillWithOval(driver ovaldb.DB, r *models.ScanResult) (nCVEs int, err error) {
var ovalClient oval.Client
var ovalFamily string
// TODO
switch r.Family {
case c.Debian:
ovalClient = oval.NewDebian()
@@ -184,60 +246,367 @@ func FillWithOval(r *models.ScanResult) (err error) {
ovalClient = oval.NewAlpine()
ovalFamily = c.Alpine
case c.Amazon, c.Raspbian, c.FreeBSD, c.Windows:
return nil
return 0, nil
case c.ServerTypePseudo:
return nil
return 0, nil
default:
if r.Family == "" {
return fmt.Errorf("Probably an error occurred during scanning. Check the error message")
return 0, fmt.Errorf("Probably an error occurred during scanning. Check the error message")
}
return fmt.Errorf("OVAL for %s is not implemented yet", r.Family)
return 0, fmt.Errorf("OVAL for %s is not implemented yet", r.Family)
}
util.Log.Debugf("Check whether oval is already fetched: %s %s",
if !ovalClient.IsFetchViaHTTP() && driver == nil {
return 0, nil
}
if err = driver.NewOvalDB(ovalFamily); err != nil {
return 0, fmt.Errorf("Failed to New Oval DB. err: %s", err)
}
util.Log.Debugf("Check whether oval fetched: %s %s",
ovalFamily, r.Release)
ok, err := ovalClient.CheckIfOvalFetched(ovalFamily, r.Release)
ok, err := ovalClient.CheckIfOvalFetched(driver, ovalFamily, r.Release)
if err != nil {
return err
return 0, err
}
if !ok {
util.Log.Warnf("OVAL entries of %s %s are not found. It's recommended to use OVAL to improve scanning accuracy. For details, see https://github.com/kotakanbe/goval-dictionary#usage , Then report with --ovaldb-path or --ovaldb-url flag", ovalFamily, r.Release)
return nil
return 0, nil
}
_, err = ovalClient.CheckIfOvalFresh(ovalFamily, r.Release)
_, err = ovalClient.CheckIfOvalFresh(driver, ovalFamily, r.Release)
if err != nil {
return err
return 0, err
}
if err := ovalClient.FillWithOval(r); err != nil {
return err
}
return nil
return ovalClient.FillWithOval(driver, r)
}
func fillVulnByCpeNames(cpeNames []string, scannedVulns models.VulnInfos) error {
for _, name := range cpeNames {
details, err := CveClient.FetchCveDetailsByCpeName(name)
// FillWithGost fills CVEs with gost dataabase
// https://github.com/knqyf263/gost
func FillWithGost(driver gostdb.DB, r *models.ScanResult) (nCVEs int, err error) {
gostClient := gost.NewClient(r.Family)
// TODO chekc if fetched
// TODO chekc if fresh enough
return gostClient.FillWithGost(driver, r)
}
func fillVulnByCpeURIs(driver cvedb.DB, r *models.ScanResult, cpeURIs []string) (nCVEs int, err error) {
for _, name := range cpeURIs {
details, err := CveClient.FetchCveDetailsByCpeName(driver, name)
if err != nil {
return err
return 0, err
}
for _, detail := range details {
if val, ok := scannedVulns[detail.CveID]; ok {
names := val.CpeNames
if val, ok := r.ScannedCves[detail.CveID]; ok {
names := val.CpeURIs
names = util.AppendIfMissing(names, name)
val.CpeNames = names
val.Confidence = models.CpeNameMatch
scannedVulns[detail.CveID] = val
val.CpeURIs = names
val.Confidences.AppendIfMissing(models.CpeNameMatch)
r.ScannedCves[detail.CveID] = val
} else {
v := models.VulnInfo{
CveID: detail.CveID,
CpeNames: []string{name},
Confidence: models.CpeNameMatch,
CveID: detail.CveID,
CpeURIs: []string{name},
Confidences: models.Confidences{models.CpeNameMatch},
}
scannedVulns[detail.CveID] = v
r.ScannedCves[detail.CveID] = v
nCVEs++
}
}
}
return nil
return nCVEs, nil
}
func fillCweDict(r *models.ScanResult) {
uniqCweIDMap := map[string]bool{}
for _, vinfo := range r.ScannedCves {
for _, cont := range vinfo.CveContents {
for _, id := range cont.CweIDs {
if strings.HasPrefix(id, "CWE-") {
id = strings.TrimPrefix(id, "CWE-")
uniqCweIDMap[id] = true
}
}
}
}
// TODO check the format of CWEID, clean CWEID
// JVN, NVD XML, JSON, OVALs
dict := map[string]models.CweDictEntry{}
for id := range uniqCweIDMap {
entry := models.CweDictEntry{}
if e, ok := cwe.CweDictEn[id]; ok {
if rank, ok := cwe.OwaspTopTen2017[id]; ok {
entry.OwaspTopTen2017 = rank
}
entry.En = &e
} else {
util.Log.Debugf("CWE-ID %s is not found in English CWE Dict", id)
entry.En = &cwe.Cwe{CweID: id}
}
if c.Conf.Lang == "ja" {
if e, ok := cwe.CweDictJa[id]; ok {
if rank, ok := cwe.OwaspTopTen2017[id]; ok {
entry.OwaspTopTen2017 = rank
}
entry.Ja = &e
} else {
util.Log.Debugf("CWE-ID %s is not found in Japanese CWE Dict", id)
entry.Ja = &cwe.Cwe{CweID: id}
}
}
dict[id] = entry
}
r.CweDict = dict
return
}
const reUUID = "[\\da-f]{8}-[\\da-f]{4}-[\\da-f]{4}-[\\da-f]{4}-[\\da-f]{12}"
// EnsureUUIDs generate a new UUID of the scan target server if UUID is not assigned yet.
// And then set the generated UUID to config.toml and scan results.
func EnsureUUIDs(configPath string, results models.ScanResults) error {
// Sort Host->Container
sort.Slice(results, func(i, j int) bool {
if results[i].ServerName == results[j].ServerName {
return results[i].Container.ContainerID < results[j].Container.ContainerID
}
return results[i].ServerName < results[j].ServerName
})
for i, r := range results {
server := c.Conf.Servers[r.ServerName]
if server.UUIDs == nil {
server.UUIDs = map[string]string{}
}
name := ""
if r.IsContainer() {
name = fmt.Sprintf("%s@%s", r.Container.Name, r.ServerName)
// Scanning with the -containers-only flag at scan time, the UUID of Container Host may not be generated,
// so check it. Otherwise create a UUID of the Container Host and set it.
serverUUID := ""
if id, ok := server.UUIDs[r.ServerName]; !ok {
serverUUID = uuid.GenerateUUID()
} else {
matched, err := regexp.MatchString(reUUID, id)
if !matched || err != nil {
serverUUID = uuid.GenerateUUID()
}
}
if serverUUID != "" {
server.UUIDs[r.ServerName] = serverUUID
}
} else {
name = r.ServerName
}
if id, ok := server.UUIDs[name]; ok {
matched, err := regexp.MatchString(reUUID, id)
if !matched || err != nil {
util.Log.Warnf("UUID is invalid. Re-generate UUID %s: %s", id, err)
} else {
if r.IsContainer() {
results[i].Container.UUID = id
results[i].ServerUUID = server.UUIDs[r.ServerName]
} else {
results[i].ServerUUID = id
}
// continue if the UUID has already assigned and valid
continue
}
}
// Generate a new UUID and set to config and scan result
id := uuid.GenerateUUID()
server.UUIDs[name] = id
server = cleanForTOMLEncoding(server, c.Conf.Default)
c.Conf.Servers[r.ServerName] = server
if r.IsContainer() {
results[i].Container.UUID = id
results[i].ServerUUID = server.UUIDs[r.ServerName]
} else {
results[i].ServerUUID = id
}
}
for name, server := range c.Conf.Servers {
server = cleanForTOMLEncoding(server, c.Conf.Default)
c.Conf.Servers[name] = server
}
email := &c.Conf.EMail
if email.SMTPAddr == "" {
email = nil
}
slack := &c.Conf.Slack
if slack.HookURL == "" {
slack = nil
}
cveDict := &c.Conf.CveDict
ovalDict := &c.Conf.OvalDict
gost := &c.Conf.Gost
http := &c.Conf.HTTP
if http.URL == "" {
http = nil
}
syslog := &c.Conf.Syslog
if syslog.Host == "" {
syslog = nil
}
aws := &c.Conf.AWS
if aws.S3Bucket == "" {
aws = nil
}
azure := &c.Conf.Azure
if azure.AccountName == "" {
azure = nil
}
stride := &c.Conf.Stride
if stride.HookURL == "" {
stride = nil
}
hipChat := &c.Conf.HipChat
if hipChat.AuthToken == "" {
hipChat = nil
}
chatWork := &c.Conf.ChatWork
if chatWork.APIToken == "" {
chatWork = nil
}
saas := &c.Conf.Saas
if saas.GroupID == 0 {
saas = nil
}
c := struct {
CveDict *c.GoCveDictConf `toml:"cveDict"`
OvalDict *c.GovalDictConf `toml:"ovalDict"`
Gost *c.GostConf `toml:"gost"`
Slack *c.SlackConf `toml:"slack"`
Email *c.SMTPConf `toml:"email"`
HTTP *c.HTTPConf `toml:"http"`
Syslog *c.SyslogConf `toml:"syslog"`
AWS *c.AWS `toml:"aws"`
Azure *c.Azure `toml:"azure"`
Stride *c.StrideConf `toml:"stride"`
HipChat *c.HipChatConf `toml:"hipChat"`
ChatWork *c.ChatWorkConf `toml:"chatWork"`
Saas *c.SaasConf `toml:"saas"`
Default c.ServerInfo `toml:"default"`
Servers map[string]c.ServerInfo `toml:"servers"`
}{
CveDict: cveDict,
OvalDict: ovalDict,
Gost: gost,
Slack: slack,
Email: email,
HTTP: http,
Syslog: syslog,
AWS: aws,
Azure: azure,
Stride: stride,
HipChat: hipChat,
ChatWork: chatWork,
Saas: saas,
Default: c.Conf.Default,
Servers: c.Conf.Servers,
}
// rename the current config.toml to config.toml.bak
info, err := os.Lstat(configPath)
if err != nil {
return fmt.Errorf("Failed to lstat %s: %s", configPath, err)
}
realPath := configPath
if info.Mode()&os.ModeSymlink == os.ModeSymlink {
if realPath, err = os.Readlink(configPath); err != nil {
return fmt.Errorf("Failed to Read link %s: %s", configPath, err)
}
}
if err := os.Rename(realPath, realPath+".bak"); err != nil {
return fmt.Errorf("Failed to rename %s: %s", configPath, err)
}
var buf bytes.Buffer
if err := toml.NewEncoder(&buf).Encode(c); err != nil {
return fmt.Errorf("Failed to encode to toml: %s", err)
}
str := strings.Replace(buf.String(), "\n [", "\n\n [", -1)
str = fmt.Sprintf("%s\n\n%s",
"# See REAME for details: https://vuls.io/docs/en/usage-settings.html",
str)
return ioutil.WriteFile(realPath, []byte(str), 0600)
}
func cleanForTOMLEncoding(server c.ServerInfo, def c.ServerInfo) c.ServerInfo {
if reflect.DeepEqual(server.Optional, def.Optional) {
server.Optional = nil
}
if def.User == server.User {
server.User = ""
}
if def.Host == server.Host {
server.Host = ""
}
if def.Port == server.Port {
server.Port = ""
}
if def.KeyPath == server.KeyPath {
server.KeyPath = ""
}
if reflect.DeepEqual(server.ScanMode, def.ScanMode) {
server.ScanMode = nil
}
if def.Type == server.Type {
server.Type = ""
}
if reflect.DeepEqual(server.CpeNames, def.CpeNames) {
server.CpeNames = nil
}
if def.OwaspDCXMLPath == server.OwaspDCXMLPath {
server.OwaspDCXMLPath = ""
}
if reflect.DeepEqual(server.IgnoreCves, def.IgnoreCves) {
server.IgnoreCves = nil
}
if reflect.DeepEqual(server.Enablerepo, def.Enablerepo) {
server.Enablerepo = nil
}
for k, v := range def.Optional {
if vv, ok := server.Optional[k]; ok && v == vv {
delete(server.Optional, k)
}
}
return server
}

24
report/s3.go
View File

@@ -41,10 +41,10 @@ type S3Writer struct{}
func getS3() *s3.S3 {
Config := &aws.Config{
Region: aws.String(c.Conf.AwsRegion),
Region: aws.String(c.Conf.AWS.Region),
Credentials: credentials.NewChainCredentials([]credentials.Provider{
&credentials.EnvProvider{},
&credentials.SharedCredentialsProvider{Filename: "", Profile: c.Conf.AwsProfile},
&credentials.SharedCredentialsProvider{Filename: "", Profile: c.Conf.AWS.Profile},
&ec2rolecreds.EC2RoleProvider{Client: ec2metadata.New(session.New())},
}),
}
@@ -82,9 +82,9 @@ func (w S3Writer) Write(rs ...models.ScanResult) (err error) {
}
}
if c.Conf.FormatShortText {
if c.Conf.FormatList {
k := key + "_short.txt"
text := formatShortPlainText(r)
text := formatList(r)
if err := putObject(svc, k, []byte(text)); err != nil {
return err
}
@@ -120,12 +120,12 @@ func CheckIfBucketExists() error {
if err != nil {
return fmt.Errorf(
"Failed to list buckets. err: %s, profile: %s, region: %s",
err, c.Conf.AwsProfile, c.Conf.AwsRegion)
err, c.Conf.AWS.Profile, c.Conf.AWS.Region)
}
found := false
for _, bucket := range result.Buckets {
if *bucket.Name == c.Conf.S3Bucket {
if *bucket.Name == c.Conf.AWS.S3Bucket {
found = true
break
}
@@ -133,7 +133,7 @@ func CheckIfBucketExists() error {
if !found {
return fmt.Errorf(
"Failed to find the buckets. profile: %s, region: %s, bucket: %s",
c.Conf.AwsProfile, c.Conf.AwsRegion, c.Conf.S3Bucket)
c.Conf.AWS.Profile, c.Conf.AWS.Region, c.Conf.AWS.S3Bucket)
}
return nil
}
@@ -148,18 +148,18 @@ func putObject(svc *s3.S3, k string, b []byte) error {
}
putObjectInput := &s3.PutObjectInput{
Bucket: aws.String(c.Conf.S3Bucket),
Key: aws.String(path.Join(c.Conf.S3ResultsDir, k)),
Bucket: aws.String(c.Conf.AWS.S3Bucket),
Key: aws.String(path.Join(c.Conf.AWS.S3ResultsDir, k)),
Body: bytes.NewReader(b),
}
if c.Conf.S3ServerSideEncryption != "" {
putObjectInput.ServerSideEncryption = aws.String(c.Conf.S3ServerSideEncryption)
if c.Conf.AWS.S3ServerSideEncryption != "" {
putObjectInput.ServerSideEncryption = aws.String(c.Conf.AWS.S3ServerSideEncryption)
}
if _, err := svc.PutObject(putObjectInput); err != nil {
return fmt.Errorf("Failed to upload data to %s/%s, %s",
c.Conf.S3Bucket, k, err)
c.Conf.AWS.S3Bucket, k, err)
}
return nil
}

153
report/saas.go Normal file
View File

@@ -0,0 +1,153 @@
/* Vuls - Vulnerability Scanner
Copyright (C) 2016 Future Corporation , Japan.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package report
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"net/url"
"path"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
"github.com/aws/aws-sdk-go/service/sts"
c "github.com/future-architect/vuls/config"
"github.com/future-architect/vuls/models"
"github.com/future-architect/vuls/util"
)
// SaasWriter writes results to SaaS
type SaasWriter struct{}
// TempCredential : TempCredential
type TempCredential struct {
Credential *sts.Credentials `json:"Credential"`
S3Bucket string `json:"S3Bucket"`
S3ResultsDir string `json:"S3ResultsDir"`
}
type payload struct {
GroupID int `json:"GroupID"`
Token string `json:"Token"`
}
// UploadSaas : UploadSaas
func (w SaasWriter) Write(rs ...models.ScanResult) (err error) {
// dir string, configPath string, config *c.Config
if len(rs) == 0 {
return nil
}
payload := payload{
GroupID: c.Conf.Saas.GroupID,
Token: c.Conf.Saas.Token,
}
var body []byte
if body, err = json.Marshal(payload); err != nil {
return fmt.Errorf("Failed to Marshal to JSON: %s", err)
}
var req *http.Request
if req, err = http.NewRequest("POST", c.Conf.Saas.URL, bytes.NewBuffer(body)); err != nil {
return err
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
proxy := c.Conf.HTTPProxy
var client http.Client
if proxy != "" {
proxyURL, _ := url.Parse(proxy)
client = http.Client{
Transport: &http.Transport{
Proxy: http.ProxyURL(proxyURL),
},
}
} else {
client = http.Client{}
}
var resp *http.Response
if resp, err = client.Do(req); err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
return fmt.Errorf("Failed to get Credential. Request JSON : %s,", string(body))
}
var t []byte
if t, err = ioutil.ReadAll(resp.Body); err != nil {
return err
}
var tempCredential TempCredential
if err = json.Unmarshal(t, &tempCredential); err != nil {
return fmt.Errorf("Failed to unmarshal saas credential file. err : %s", err)
}
credential := credentials.NewStaticCredentialsFromCreds(credentials.Value{
AccessKeyID: *tempCredential.Credential.AccessKeyId,
SecretAccessKey: *tempCredential.Credential.SecretAccessKey,
SessionToken: *tempCredential.Credential.SessionToken,
})
var sess *session.Session
if sess, err = session.NewSession(&aws.Config{
Credentials: credential,
Region: aws.String("ap-northeast-1"),
}); err != nil {
return fmt.Errorf("Failed to new aws session. err : %s", err)
}
svc := s3.New(sess)
for _, r := range rs {
s3Key := renameKeyNameUTC(r.ScannedAt, r.ServerUUID, r.Container)
var b []byte
if b, err = json.Marshal(r); err != nil {
return fmt.Errorf("Failed to Marshal to JSON: %s", err)
}
util.Log.Infof("Uploading...: ServerName: %s, ", r.ServerName)
putObjectInput := &s3.PutObjectInput{
Bucket: aws.String(tempCredential.S3Bucket),
Key: aws.String(path.Join(tempCredential.S3ResultsDir, s3Key)),
Body: bytes.NewReader(b),
}
if _, err := svc.PutObject(putObjectInput); err != nil {
return fmt.Errorf("Failed to upload data to %s/%s, %s",
tempCredential.S3Bucket, s3Key, err)
}
}
return nil
}
func renameKeyNameUTC(scannedAt time.Time, uuid string, container models.Container) string {
timestr := scannedAt.UTC().Format(time.RFC3339)
if len(container.ContainerID) == 0 {
return fmt.Sprintf("%s/%s.json", timestr, uuid)
}
return fmt.Sprintf("%s/%s@%s.json", timestr, container.UUID, uuid)
}

76
report/slack.go
View File

@@ -177,18 +177,20 @@ func msgText(r models.ScanResult) string {
serverInfo := fmt.Sprintf("*%s*", r.ServerInfo())
if 0 < len(r.Errors) {
return fmt.Sprintf("%s\n%s\n%s\n%s\nError: %s",
return fmt.Sprintf("%s\n%s\n%s\n%s\n%s\nError: %s",
notifyUsers,
serverInfo,
r.ScannedCves.FormatCveSummary(),
r.Packages.FormatUpdatablePacksSummary(),
r.ScannedCves.FormatFixedStatus(r.Packages),
r.FormatUpdatablePacksSummary(),
r.Errors)
}
return fmt.Sprintf("%s\n%s\n%s\n%s",
return fmt.Sprintf("%s\n%s\n%s\n%s\n%s",
notifyUsers,
serverInfo,
r.ScannedCves.FormatCveSummary(),
r.Packages.FormatUpdatablePacksSummary())
r.ScannedCves.FormatFixedStatus(r.Packages),
r.FormatUpdatablePacksSummary())
}
func toSlackAttachments(r models.ScanResult) (attaches []slack.Attachment) {
@@ -203,7 +205,7 @@ func toSlackAttachments(r models.ScanResult) (attaches []slack.Attachment) {
curent = append(curent, affected.Name)
}
}
for _, n := range vinfo.CpeNames {
for _, n := range vinfo.CpeURIs {
curent = append(curent, n)
}
@@ -219,14 +221,14 @@ func toSlackAttachments(r models.ScanResult) (attaches []slack.Attachment) {
new = append(new, "?")
}
}
for range vinfo.CpeNames {
for range vinfo.CpeURIs {
new = append(new, "?")
}
a := slack.Attachment{
Title: vinfo.CveID,
TitleLink: "https://nvd.nist.gov/vuln/detail/" + vinfo.CveID,
Text: attachmentText(vinfo, r.Family),
Text: attachmentText(vinfo, r.Family, r.CweDict, r.Packages),
MarkdownIn: []string{"text", "pretext"},
Fields: []slack.AttachmentField{
{
@@ -241,7 +243,7 @@ func toSlackAttachments(r models.ScanResult) (attaches []slack.Attachment) {
Short: true,
},
},
Color: color(vinfo.MaxCvssScore().Value.Score),
Color: cvssColor(vinfo.MaxCvssScore().Value.Score),
}
attaches = append(attaches, a)
}
@@ -249,7 +251,7 @@ func toSlackAttachments(r models.ScanResult) (attaches []slack.Attachment) {
}
// https://api.slack.com/docs/attachments
func color(cvssScore float64) string {
func cvssColor(cvssScore float64) string {
switch {
case 7 <= cvssScore:
return "danger"
@@ -262,10 +264,15 @@ func color(cvssScore float64) string {
}
}
func attachmentText(vinfo models.VulnInfo, osFamily string) string {
func attachmentText(vinfo models.VulnInfo, osFamily string, cweDict map[string]models.CweDictEntry, packs models.Packages) string {
maxCvss := vinfo.MaxCvssScore()
vectors := []string{}
for _, cvss := range vinfo.Cvss2Scores() {
scores := append(vinfo.Cvss3Scores(), vinfo.Cvss2Scores(osFamily)...)
for _, cvss := range scores {
if cvss.Value.Severity == "" {
continue
}
calcURL := ""
switch cvss.Value.Type {
case models.CVSS2:
@@ -279,9 +286,10 @@ func attachmentText(vinfo models.VulnInfo, osFamily string) string {
}
if cont, ok := vinfo.CveContents[cvss.Type]; ok {
v := fmt.Sprintf("<%s|%s> (<%s|%s>)",
v := fmt.Sprintf("<%s|%s> %s (<%s|%s>)",
calcURL,
cvss.Value.Format(),
fmt.Sprintf("%3.1f/%s", cvss.Value.Score, cvss.Value.Vector),
cvss.Value.Severity,
cont.SourceLink,
cvss.Type)
vectors = append(vectors, v)
@@ -294,9 +302,10 @@ func attachmentText(vinfo models.VulnInfo, osFamily string) string {
v, k))
}
v := fmt.Sprintf("<%s|%s> (%s)",
v := fmt.Sprintf("<%s|%s> %s (%s)",
calcURL,
cvss.Value.Format(),
fmt.Sprintf("%3.1f/%s", cvss.Value.Score, cvss.Value.Vector),
cvss.Value.Severity,
strings.Join(links, ", "))
vectors = append(vectors, v)
}
@@ -308,27 +317,42 @@ func attachmentText(vinfo models.VulnInfo, osFamily string) string {
severity = "?"
}
return fmt.Sprintf("*%4.1f (%s)* %s\n%s\n```%s```",
nwvec := vinfo.AttackVector()
if nwvec == "Network" || nwvec == "remote" {
nwvec = fmt.Sprintf("*%s*", nwvec)
}
mitigation := ""
if vinfo.Mitigations(osFamily)[0].Type != models.Unknown {
mitigation = fmt.Sprintf("\nMitigation:\n```%s```\n",
vinfo.Mitigations(osFamily)[0].Value)
}
return fmt.Sprintf("*%4.1f (%s)* %s %s\n%s\n```\n%s\n```%s\n%s\n",
maxCvss.Value.Score,
severity,
cweIDs(vinfo, osFamily),
nwvec,
vinfo.PatchStatus(packs),
strings.Join(vectors, "\n"),
vinfo.Summaries(config.Conf.Lang, osFamily)[0].Value,
mitigation,
cweIDs(vinfo, osFamily, cweDict),
)
}
func cweIDs(vinfo models.VulnInfo, osFamily string) string {
func cweIDs(vinfo models.VulnInfo, osFamily string, cweDict models.CweDict) string {
links := []string{}
for _, cwe := range vinfo.CveContents.CweIDs(osFamily) {
if config.Conf.Lang == "ja" {
links = append(links, fmt.Sprintf("<%s|%s>",
cweJvnURL(cwe.Value), cwe.Value))
} else {
links = append(links, fmt.Sprintf("<%s|%s>",
cweURL(cwe.Value), cwe.Value))
for _, c := range vinfo.CveContents.UniqCweIDs(osFamily) {
name, url, top10Rank, top10URL := cweDict.Get(c.Value, osFamily)
line := ""
if top10Rank != "" {
line = fmt.Sprintf("<%s|[OWASP Top %s]>",
top10URL, top10Rank)
}
links = append(links, fmt.Sprintf("%s <%s|%s>: %s",
line, url, c.Value, name))
}
return strings.Join(links, " / ")
return strings.Join(links, "\n")
}
// See testcase

4
report/stdout.go
View File

@@ -44,9 +44,9 @@ func (w StdoutWriter) Write(rs ...models.ScanResult) error {
fmt.Print("\n")
}
if c.Conf.FormatShortText {
if c.Conf.FormatList {
for _, r := range rs {
fmt.Println(formatShortPlainText(r))
fmt.Println(formatList(r))
}
}

7
report/syslog.go
View File

@@ -76,7 +76,7 @@ func (w SyslogWriter) encodeSyslog(result models.ScanResult) (messages []string)
kvPairs = append(kvPairs, fmt.Sprintf(`packages="%s"`, pkgs))
kvPairs = append(kvPairs, fmt.Sprintf(`cve_id="%s"`, cveID))
for _, cvss := range vinfo.Cvss2Scores() {
for _, cvss := range vinfo.Cvss2Scores(result.Family) {
kvPairs = append(kvPairs, fmt.Sprintf(`cvss_score_%s_v2="%.2f"`, cvss.Type, cvss.Value.Score))
kvPairs = append(kvPairs, fmt.Sprintf(`cvss_vector_%s_v2="%s"`, cvss.Type, cvss.Value.Vector))
}
@@ -86,8 +86,9 @@ func (w SyslogWriter) encodeSyslog(result models.ScanResult) (messages []string)
kvPairs = append(kvPairs, fmt.Sprintf(`cvss_vector_%s_v3="%s"`, cvss.Type, cvss.Value.Vector))
}
if content, ok := vinfo.CveContents[models.NVD]; ok {
kvPairs = append(kvPairs, fmt.Sprintf(`cwe_id="%s"`, content.CweID))
if content, ok := vinfo.CveContents[models.NvdXML]; ok {
cwes := strings.Join(content.CweIDs, ",")
kvPairs = append(kvPairs, fmt.Sprintf(`cwe_ids="%s"`, cwes))
if config.Conf.Syslog.Verbose {
kvPairs = append(kvPairs, fmt.Sprintf(`source_link="%s"`, content.SourceLink))
kvPairs = append(kvPairs, fmt.Sprintf(`summary="%s"`, content.Summary))

15
report/syslog_test.go
View File

@@ -33,10 +33,11 @@ func TestSyslogWriterEncodeSyslog(t *testing.T) {
models.PackageStatus{Name: "pkg4"},
},
CveContents: models.CveContents{
models.NVD: models.CveContent{
Cvss2Score: 5.0,
Cvss2Vector: "AV:L/AC:L/Au:N/C:N/I:N/A:C",
CweID: "CWE-20",
models.NvdXML: models.CveContent{
Cvss2Score: 5.0,
Cvss2Vector: "AV:L/AC:L/Au:N/C:N/I:N/A:C",
Cvss2Severity: "MEDIUM",
CweIDs: []string{"CWE-20"},
},
},
},
@@ -44,7 +45,7 @@ func TestSyslogWriterEncodeSyslog(t *testing.T) {
},
expectedMessages: []string{
`scanned_at="2018-06-13 16:10:00 +0000 UTC" server_name="teste01" os_family="ubuntu" os_release="16.04" ipv4_addr="192.168.0.1,10.0.2.15" ipv6_addr="" packages="pkg1,pkg2" cve_id="CVE-2017-0001"`,
`scanned_at="2018-06-13 16:10:00 +0000 UTC" server_name="teste01" os_family="ubuntu" os_release="16.04" ipv4_addr="192.168.0.1,10.0.2.15" ipv6_addr="" packages="pkg3,pkg4" cve_id="CVE-2017-0002" cvss_score_nvd_v2="5.00" cvss_vector_nvd_v2="AV:L/AC:L/Au:N/C:N/I:N/A:C" cwe_id="CWE-20"`,
`scanned_at="2018-06-13 16:10:00 +0000 UTC" server_name="teste01" os_family="ubuntu" os_release="16.04" ipv4_addr="192.168.0.1,10.0.2.15" ipv6_addr="" packages="pkg3,pkg4" cve_id="CVE-2017-0002" cvss_score_nvdxml_v2="5.00" cvss_vector_nvdxml_v2="AV:L/AC:L/Au:N/C:N/I:N/A:C" cwe_ids="CWE-20"`,
},
},
{
@@ -63,7 +64,7 @@ func TestSyslogWriterEncodeSyslog(t *testing.T) {
models.RedHat: models.CveContent{
Cvss3Score: 5.0,
Cvss3Vector: "AV:L/AC:L/Au:N/C:N/I:N/A:C",
CweID: "CWE-284",
CweIDs: []string{"CWE-284"},
Title: "RHSA-2017:0001: pkg5 security update (Important)",
},
},
@@ -103,7 +104,7 @@ func TestSyslogWriterEncodeSyslog(t *testing.T) {
for j, m := range messages {
e := tt.expectedMessages[j]
if e != m {
t.Errorf("test: %d, Messsage %d: expected %s, actual %s", i, j, e, m)
t.Errorf("test: %d, Messsage %d: \nexpected %s \nactual %s", i, j, e, m)
}
}
}

222
report/tui.go
View File

@@ -32,7 +32,6 @@ import (
"github.com/google/subcommands"
"github.com/gosuri/uitable"
"github.com/jroimartin/gocui"
log "github.com/sirupsen/logrus"
)
var scanResults models.ScanResults
@@ -55,7 +54,7 @@ func RunTui(results models.ScanResults) subcommands.ExitStatus {
// g, err := gocui.NewGui(gocui.OutputNormal)
g := gocui.NewGui()
if err := g.Init(); err != nil {
log.Errorf("%s", err)
util.Log.Errorf("%s", err)
return subcommands.ExitFailure
}
defer g.Close()
@@ -63,7 +62,7 @@ func RunTui(results models.ScanResults) subcommands.ExitStatus {
g.SetLayout(layout)
// g.SetManagerFunc(layout)
if err := keybindings(g); err != nil {
log.Errorf("%s", err)
util.Log.Errorf("%s", err)
return subcommands.ExitFailure
}
g.SelBgColor = gocui.ColorGreen
@@ -72,7 +71,7 @@ func RunTui(results models.ScanResults) subcommands.ExitStatus {
if err := g.MainLoop(); err != nil {
g.Close()
log.Errorf("%s", err)
util.Log.Errorf("%s", err)
os.Exit(1)
}
return subcommands.ExitSuccess
@@ -468,10 +467,7 @@ func changeHost(g *gocui.Gui, v *gocui.View) error {
if err := setDetailLayout(g); err != nil {
return err
}
if err := setChangelogLayout(g); err != nil {
return err
}
return nil
return setChangelogLayout(g)
}
func redrawDetail(g *gocui.Gui) error {
@@ -479,10 +475,7 @@ func redrawDetail(g *gocui.Gui) error {
return err
}
if err := setDetailLayout(g); err != nil {
return err
}
return nil
return setDetailLayout(g)
}
func redrawChangelog(g *gocui.Gui) error {
@@ -490,10 +483,7 @@ func redrawChangelog(g *gocui.Gui) error {
return err
}
if err := setChangelogLayout(g); err != nil {
return err
}
return nil
return setChangelogLayout(g)
}
func getLine(g *gocui.Gui, v *gocui.View) error {
@@ -545,10 +535,7 @@ func delMsg(g *gocui.Gui, v *gocui.View) error {
if err := g.DeleteView("msg"); err != nil {
return err
}
if err := g.SetCurrentView("summary"); err != nil {
return err
}
return nil
return g.SetCurrentView("summary")
}
func quit(g *gocui.Gui, v *gocui.View) error {
@@ -565,11 +552,7 @@ func layout(g *gocui.Gui) error {
if err := setDetailLayout(g); err != nil {
return err
}
if err := setChangelogLayout(g); err != nil {
return err
}
return nil
return setChangelogLayout(g)
}
func debug(g *gocui.Gui, str string) error {
@@ -615,7 +598,7 @@ func setSummaryLayout(g *gocui.Gui) error {
return err
}
lines := summaryLines()
lines := summaryLines(currentScanResult)
fmt.Fprintf(v, lines)
v.Highlight = true
@@ -625,37 +608,42 @@ func setSummaryLayout(g *gocui.Gui) error {
return nil
}
func summaryLines() string {
func summaryLines(r models.ScanResult) string {
stable := uitable.New()
stable.MaxColWidth = 1000
stable.Wrap = false
if len(currentScanResult.Errors) != 0 {
if len(r.Errors) != 0 {
return "Error: Scan with --debug to view the details"
}
indexFormat := ""
if len(currentScanResult.ScannedCves) < 10 {
if len(r.ScannedCves) < 10 {
indexFormat = "[%1d]"
} else if len(currentScanResult.ScannedCves) < 100 {
} else if len(r.ScannedCves) < 100 {
indexFormat = "[%2d]"
} else {
indexFormat = "[%3d]"
}
for i, vinfo := range vinfos {
summary := vinfo.Titles(
config.Conf.Lang, currentScanResult.Family)[0].Value
cvssScore := fmt.Sprintf("| %4.1f",
vinfo.MaxCvssScore().Value.Score)
for i, vinfo := range r.ScannedCves.ToSortedSlice() {
max := vinfo.MaxCvssScore().Value.Score
cvssScore := "| "
if 0 < max {
cvssScore = fmt.Sprintf("| %4.1f", max)
}
packname := vinfo.AffectedPackages.FormatTuiSummary()
packname += strings.Join(vinfo.CpeURIs, ", ")
var cols []string
cols = []string{
fmt.Sprintf(indexFormat, i+1),
vinfo.CveID,
cvssScore,
fmt.Sprintf("| %3d |", vinfo.Confidence.Score),
summary,
cvssScore + " |",
fmt.Sprintf("%8s |", vinfo.AttackVector()),
fmt.Sprintf("%7s |", vinfo.PatchStatus(r.Packages)),
packname,
}
icols := make([]interface{}, len(cols))
for j := range cols {
@@ -695,16 +683,12 @@ func setDetailLayout(g *gocui.Gui) error {
}
func setChangelogLayout(g *gocui.Gui) error {
maxX, maxY := g.Size()
summaryView, err := g.View("summary")
if err != nil {
return err
}
_, cy := summaryView.Cursor()
_, oy := summaryView.Origin()
currentVinfo = cy + oy
maxX, maxY := g.Size()
if v, err := g.SetView("changelog", int(float64(maxX)*0.5), int(float64(maxY)*0.2), maxX, maxY); err != nil {
if err != gocui.ErrUnknownView {
return err
@@ -713,17 +697,56 @@ func setChangelogLayout(g *gocui.Gui) error {
return nil
}
lines := []string{}
lines := []string{
"Affected Packages, Processes",
"============================",
}
_, cy := summaryView.Cursor()
_, oy := summaryView.Origin()
currentVinfo = cy + oy
vinfo := vinfos[currentVinfo]
vinfo.AffectedPackages.Sort()
for _, affected := range vinfo.AffectedPackages {
// packages detected by OVAL may not be actually installed
if pack, ok := currentScanResult.Packages[affected.Name]; ok {
lines = append(lines,
"* "+pack.FormatVersionFromTo(
affected.NotFixedYet, affected.FixState))
if len(pack.AffectedProcs) != 0 {
for _, p := range pack.AffectedProcs {
lines = append(lines, fmt.Sprintf(" * PID: %s %s", p.PID, p.Name))
}
} else {
// lines = append(lines, fmt.Sprintf(" * No affected process"))
}
}
}
sort.Strings(vinfo.CpeURIs)
for _, uri := range vinfo.CpeURIs {
lines = append(lines, "* "+uri)
}
for _, adv := range vinfo.DistroAdvisories {
lines = append(lines, "\n",
"Advisories",
"==========",
)
lines = append(lines, adv.Format())
}
for _, affected := range vinfo.AffectedPackages {
pack := currentScanResult.Packages[affected.Name]
for _, p := range currentScanResult.Packages {
if pack.Name == p.Name {
lines = append(lines, p.FormatChangelog(), "\n")
if currentScanResult.IsDeepScanMode() {
lines = append(lines, "\n",
"ChangeLogs",
"==========",
)
for _, affected := range vinfo.AffectedPackages {
pack := currentScanResult.Packages[affected.Name]
for _, p := range currentScanResult.Packages {
if pack.Name == p.Name {
lines = append(lines, p.FormatChangelog(), "\n")
}
}
}
}
@@ -741,12 +764,13 @@ type dataForTmpl struct {
CveID string
Cvsses string
Summary string
Confidence models.Confidence
Cwes []models.CveContentStr
Mitigation string
Confidences models.Confidences
Cwes []models.CweDictEntry
Links []string
References []models.Reference
Packages []string
CpeNames []string
CpeURIs []string
PublishedDate time.Time
LastModifiedDate time.Time
}
@@ -767,20 +791,6 @@ func detailLines() (string, error) {
}
vinfo := vinfos[currentVinfo]
packsVer := []string{}
vinfo.AffectedPackages.Sort()
for _, affected := range vinfo.AffectedPackages {
// packages detected by OVAL may not be actually installed
if pack, ok := r.Packages[affected.Name]; ok {
packsVer = append(packsVer, pack.FormatVersionFromTo(affected.NotFixedYet))
}
}
sort.Strings(vinfo.CpeNames)
for _, name := range vinfo.CpeNames {
packsVer = append(packsVer, name)
}
links := []string{vinfo.CveContents.SourceLinks(
config.Conf.Lang, r.Family, vinfo.CveID)[0].Value,
vinfo.Cvss2CalcURL(),
@@ -792,35 +802,57 @@ func detailLines() (string, error) {
refs := []models.Reference{}
for _, rr := range vinfo.CveContents.References(r.Family) {
for _, ref := range rr.Value {
if ref.Source == "" {
ref.Source = "-"
}
refs = append(refs, ref)
}
}
summary := vinfo.Summaries(r.Lang, r.Family)[0]
mitigation := vinfo.Mitigations(r.Family)[0]
table := uitable.New()
table.MaxColWidth = maxColWidth
table.Wrap = true
scores := append(vinfo.Cvss3Scores(), vinfo.Cvss2Scores()...)
scores := append(vinfo.Cvss3Scores(), vinfo.Cvss2Scores(r.Family)...)
var cols []interface{}
for _, score := range scores {
if score.Value.Score == 0 && score.Value.Severity == "" {
continue
}
scoreStr := "-"
if 0 < score.Value.Score {
scoreStr = fmt.Sprintf("%3.1f", score.Value.Score)
}
scoreVec := fmt.Sprintf("%s/%s", scoreStr, score.Value.Vector)
cols = []interface{}{
scoreVec,
score.Value.Severity,
score.Value.Format(),
score.Type,
}
table.AddRow(cols...)
}
uniqCweIDs := vinfo.CveContents.UniqCweIDs(r.Family)
cwes := []models.CweDictEntry{}
for _, cweID := range uniqCweIDs {
if strings.HasPrefix(cweID.Value, "CWE-") {
if dict, ok := r.CweDict[strings.TrimPrefix(cweID.Value, "CWE-")]; ok {
cwes = append(cwes, dict)
}
}
}
data := dataForTmpl{
CveID: vinfo.CveID,
Cvsses: fmt.Sprintf("%s\n", table),
Summary: fmt.Sprintf("%s (%s)", summary.Value, summary.Type),
Confidence: vinfo.Confidence,
Cwes: vinfo.CveContents.CweIDs(r.Family),
Links: util.Distinct(links),
Packages: packsVer,
References: refs,
CveID: vinfo.CveID,
Cvsses: fmt.Sprintf("%s\n", table),
Summary: fmt.Sprintf("%s (%s)", summary.Value, summary.Type),
Mitigation: fmt.Sprintf("%s (%s)", mitigation.Value, mitigation.Type),
Confidences: vinfo.Confidences,
Cwes: cwes,
Links: util.Distinct(links),
References: refs,
}
buf := bytes.NewBuffer(nil) // create empty buffer
@@ -833,47 +865,41 @@ func detailLines() (string, error) {
const mdTemplate = `
{{.CveID}}
==============
================
CVSS Scores
--------------
-----------
{{.Cvsses }}
Summary
--------------
-----------
{{.Summary }}
Mitigation
-----------
{{.Mitigation }}
Links
--------------
-----------
{{range $link := .Links -}}
* {{$link}}
{{end}}
CWE
--------------
-----------
{{range .Cwes -}}
* {{.Value}} ({{.Type}})
* {{.En.CweID}} [{{.En.Name}}](https://cwe.mitre.org/data/definitions/{{.En.CweID}}.html)
{{end}}
Package/CPE
--------------
{{range $pack := .Packages -}}
* {{$pack}}
{{end -}}
{{range $name := .CpeNames -}}
{{range $name := .CpeURIs -}}
* {{$name}}
{{end}}
Confidence
--------------
{{.Confidence }}
-----------
{{range $confidence := .Confidences -}}
* {{$confidence.DetectionMethod}}
{{end}}
References
--------------
-----------
{{range .References -}}
* [{{.Source}}]( {{.Link}} )
* [{{.Source}}]({{.Link}})
{{end}}
`

236
report/util.go
View File

@@ -18,6 +18,7 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
package report
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
@@ -33,9 +34,10 @@ import (
"github.com/future-architect/vuls/models"
"github.com/future-architect/vuls/util"
"github.com/gosuri/uitable"
"github.com/olekukonko/tablewriter"
)
const maxColWidth = 80
const maxColWidth = 100
func formatScanSummary(rs ...models.ScanResult) string {
table := uitable.New()
@@ -47,7 +49,7 @@ func formatScanSummary(rs ...models.ScanResult) string {
cols = []interface{}{
r.FormatServerName(),
fmt.Sprintf("%s%s", r.Family, r.Release),
r.Packages.FormatUpdatablePacksSummary(),
r.FormatUpdatablePacksSummary(),
}
} else {
cols = []interface{}{
@@ -72,7 +74,8 @@ func formatOneLineSummary(rs ...models.ScanResult) string {
cols = []interface{}{
r.FormatServerName(),
r.ScannedCves.FormatCveSummary(),
r.Packages.FormatUpdatablePacksSummary(),
r.ScannedCves.FormatFixedStatus(r.Packages),
r.FormatUpdatablePacksSummary(),
}
} else {
cols = []interface{}{
@@ -86,7 +89,7 @@ func formatOneLineSummary(rs ...models.ScanResult) string {
return fmt.Sprintf("%s\n", table)
}
func formatShortPlainText(r models.ScanResult) string {
func formatList(r models.ScanResult) string {
header := r.FormatTextReportHeadedr()
if len(r.Errors) != 0 {
return fmt.Sprintf(
@@ -99,63 +102,49 @@ func formatShortPlainText(r models.ScanResult) string {
%s
No CVE-IDs are found in updatable packages.
%s
`, header, r.Packages.FormatUpdatablePacksSummary())
`, header, r.FormatUpdatablePacksSummary())
}
stable := uitable.New()
stable.MaxColWidth = maxColWidth
stable.Wrap = true
for _, vuln := range r.ScannedCves.ToSortedSlice() {
summaries := vuln.Summaries(config.Conf.Lang, r.Family)
links := vuln.CveContents.SourceLinks(
config.Conf.Lang, r.Family, vuln.CveID)
data := [][]string{}
for _, vinfo := range r.ScannedCves.ToSortedSlice() {
max := vinfo.MaxCvssScore().Value.Score
// v2max := vinfo.MaxCvss2Score().Value.Score
// v3max := vinfo.MaxCvss3Score().Value.Score
vlinks := []string{}
for name, url := range vuln.VendorLinks(r.Family) {
vlinks = append(vlinks, fmt.Sprintf("%s (%s)", url, name))
}
// packname := vinfo.AffectedPackages.FormatTuiSummary()
// packname += strings.Join(vinfo.CpeURIs, ", ")
cvsses := ""
for _, cvss := range vuln.Cvss2Scores() {
cvsses += fmt.Sprintf("%s (%s)\n", cvss.Value.Format(), cvss.Type)
}
cvsses += vuln.Cvss2CalcURL() + "\n"
for _, cvss := range vuln.Cvss3Scores() {
cvsses += fmt.Sprintf("%s (%s)\n", cvss.Value.Format(), cvss.Type)
}
if 0 < len(vuln.Cvss3Scores()) {
cvsses += vuln.Cvss3CalcURL() + "\n"
}
maxCvss := vuln.FormatMaxCvssScore()
rightCol := fmt.Sprintf(`%s
%s
---
%s
%s
%sConfidence: %v`,
maxCvss,
summaries[0].Value,
links[0].Value,
strings.Join(vlinks, "\n"),
cvsses,
// packsVer,
vuln.Confidence,
)
leftCol := fmt.Sprintf("%s", vuln.CveID)
scols := []string{leftCol, rightCol}
cols := make([]interface{}, len(scols))
for i := range cols {
cols[i] = scols[i]
}
stable.AddRow(cols...)
stable.AddRow("")
data = append(data, []string{
vinfo.CveID,
fmt.Sprintf("%4.1f", max),
// fmt.Sprintf("%4.1f", v2max),
// fmt.Sprintf("%4.1f", v3max),
fmt.Sprintf("%8s", vinfo.AttackVector()),
fmt.Sprintf("%7s", vinfo.PatchStatus(r.Packages)),
// packname,
fmt.Sprintf("https://nvd.nist.gov/vuln/detail/%s", vinfo.CveID),
})
}
return fmt.Sprintf("%s\n%s\n", header, stable)
b := bytes.Buffer{}
table := tablewriter.NewWriter(&b)
table.SetHeader([]string{
"CVE-ID",
"CVSS",
// "v3",
// "v2",
"Attack",
"Fixed",
// "Pkg",
"NVD",
})
table.SetBorder(true)
table.AppendBulk(data)
table.Render()
return fmt.Sprintf("%s\n%s", header, b.String())
}
func formatFullPlainText(r models.ScanResult) string {
func formatFullPlainText(r models.ScanResult) (lines string) {
header := r.FormatTextReportHeadedr()
if len(r.Errors) != 0 {
return fmt.Sprintf(
@@ -168,62 +157,117 @@ func formatFullPlainText(r models.ScanResult) string {
%s
No CVE-IDs are found in updatable packages.
%s
`, header, r.Packages.FormatUpdatablePacksSummary())
`, header, r.FormatUpdatablePacksSummary())
}
table := uitable.New()
table.MaxColWidth = maxColWidth
table.Wrap = true
lines = header + "\n"
for _, vuln := range r.ScannedCves.ToSortedSlice() {
table.AddRow(vuln.CveID)
table.AddRow("----------------")
table.AddRow("Max Score", vuln.FormatMaxCvssScore())
for _, cvss := range vuln.Cvss2Scores() {
table.AddRow(cvss.Type, cvss.Value.Format())
}
data := [][]string{}
data = append(data, []string{"Max Score", vuln.FormatMaxCvssScore()})
for _, cvss := range vuln.Cvss3Scores() {
table.AddRow(cvss.Type, cvss.Value.Format())
}
if 0 < len(vuln.Cvss2Scores()) {
table.AddRow("CVSSv2 Calc", vuln.Cvss2CalcURL())
}
if 0 < len(vuln.Cvss3Scores()) {
table.AddRow("CVSSv3 Calc", vuln.Cvss3CalcURL())
}
table.AddRow("Summary", vuln.Summaries(
config.Conf.Lang, r.Family)[0].Value)
links := vuln.CveContents.SourceLinks(
config.Conf.Lang, r.Family, vuln.CveID)
table.AddRow("Source", links[0].Value)
vlinks := vuln.VendorLinks(r.Family)
for name, url := range vlinks {
table.AddRow(name, url)
if cvssstr := cvss.Value.Format(); cvssstr != "" {
data = append(data, []string{string(cvss.Type), cvssstr})
}
}
for _, v := range vuln.CveContents.CweIDs(r.Family) {
table.AddRow(fmt.Sprintf("%s (%s)", v.Value, v.Type), cweURL(v.Value))
for _, cvss := range vuln.Cvss2Scores(r.Family) {
if cvssstr := cvss.Value.Format(); cvssstr != "" {
data = append(data, []string{string(cvss.Type), cvssstr})
}
}
data = append(data, []string{"Summary", vuln.Summaries(
config.Conf.Lang, r.Family)[0].Value})
mitigation := vuln.Mitigations(r.Family)[0]
if mitigation.Type != models.Unknown {
data = append(data, []string{"Mitigation", mitigation.Value})
}
cweURLs, top10URLs := []string{}, []string{}
for _, v := range vuln.CveContents.UniqCweIDs(r.Family) {
name, url, top10Rank, top10URL := r.CweDict.Get(v.Value, r.Lang)
if top10Rank != "" {
data = append(data, []string{"CWE",
fmt.Sprintf("[OWASP Top%s] %s: %s (%s)",
top10Rank, v.Value, name, v.Type)})
top10URLs = append(top10URLs, top10URL)
} else {
data = append(data, []string{"CWE", fmt.Sprintf("%s: %s (%s)",
v.Value, name, v.Type)})
}
cweURLs = append(cweURLs, url)
}
packsVer := []string{}
vuln.AffectedPackages.Sort()
for _, affected := range vuln.AffectedPackages {
if pack, ok := r.Packages[affected.Name]; ok {
packsVer = append(packsVer, pack.FormatVersionFromTo(affected.NotFixedYet))
data = append(data, []string{"Affected PKG",
pack.FormatVersionFromTo(affected.NotFixedYet, affected.FixState)})
if len(pack.AffectedProcs) != 0 {
for _, p := range pack.AffectedProcs {
data = append(data, []string{"",
fmt.Sprintf(" - PID: %s %s", p.PID, p.Name)})
}
}
}
}
sort.Strings(vuln.CpeNames)
for _, name := range vuln.CpeNames {
packsVer = append(packsVer, name)
sort.Strings(vuln.CpeURIs)
for _, name := range vuln.CpeURIs {
data = append(data, []string{"CPE", name})
}
table.AddRow("Package/CPE", strings.Join(packsVer, "\n"))
table.AddRow("Confidence", vuln.Confidence)
table.AddRow("\n")
for _, confidence := range vuln.Confidences {
data = append(data, []string{"Confidence", confidence.String()})
}
links := vuln.CveContents.SourceLinks(
config.Conf.Lang, r.Family, vuln.CveID)
data = append(data, []string{"Source", links[0].Value})
if 0 < len(vuln.Cvss2Scores(r.Family)) {
data = append(data, []string{"CVSSv2 Calc", vuln.Cvss2CalcURL()})
}
if 0 < len(vuln.Cvss3Scores()) {
data = append(data, []string{"CVSSv3 Calc", vuln.Cvss3CalcURL()})
}
vlinks := vuln.VendorLinks(r.Family)
for name, url := range vlinks {
data = append(data, []string{name, url})
}
for _, url := range cweURLs {
data = append(data, []string{"CWE", url})
}
for _, url := range top10URLs {
data = append(data, []string{"OWASP Top10", url})
}
// for _, rr := range vuln.CveContents.References(r.Family) {
// for _, ref := range rr.Value {
// data = append(data, []string{ref.Source, ref.Link})
// }
// }
b := bytes.Buffer{}
table := tablewriter.NewWriter(&b)
table.SetColWidth(80)
table.SetHeaderAlignment(tablewriter.ALIGN_LEFT)
table.SetHeader([]string{
vuln.CveID,
"",
})
table.SetBorder(true)
table.SetHeaderColor(
tablewriter.Colors{tablewriter.Normal},
tablewriter.Colors{tablewriter.Normal},
)
table.AppendBulk(data)
table.Render()
lines += b.String() + "\n"
}
return fmt.Sprintf("%s\n%s", header, table)
return
}
func cweURL(cweID string) string {
@@ -387,8 +431,8 @@ func isCveFixed(current models.VulnInfo, previous models.ScanResult) bool {
func isCveInfoUpdated(cveID string, previous, current models.ScanResult) bool {
cTypes := []models.CveContentType{
models.NVD,
models.JVN,
models.NvdXML,
models.Jvn,
models.NewCveContentType(current.Family),
}

40
report/util_test.go
View File

@@ -42,7 +42,7 @@ func TestIsCveInfoUpdated(t *testing.T) {
CveID: "CVE-2017-0001",
CveContents: models.NewCveContents(
models.CveContent{
Type: models.NVD,
Type: models.NvdXML,
CveID: "CVE-2017-0001",
LastModified: time.Time{},
},
@@ -56,7 +56,7 @@ func TestIsCveInfoUpdated(t *testing.T) {
CveID: "CVE-2017-0001",
CveContents: models.NewCveContents(
models.CveContent{
Type: models.NVD,
Type: models.NvdXML,
CveID: "CVE-2017-0001",
LastModified: time.Time{},
},
@@ -77,7 +77,7 @@ func TestIsCveInfoUpdated(t *testing.T) {
CveID: "CVE-2017-0002",
CveContents: models.NewCveContents(
models.CveContent{
Type: models.JVN,
Type: models.Jvn,
CveID: "CVE-2017-0002",
LastModified: old,
},
@@ -91,7 +91,7 @@ func TestIsCveInfoUpdated(t *testing.T) {
CveID: "CVE-2017-0002",
CveContents: models.NewCveContents(
models.CveContent{
Type: models.JVN,
Type: models.Jvn,
CveID: "CVE-2017-0002",
LastModified: old,
},
@@ -113,7 +113,7 @@ func TestIsCveInfoUpdated(t *testing.T) {
CveID: "CVE-2017-0003",
CveContents: models.NewCveContents(
models.CveContent{
Type: models.NVD,
Type: models.NvdXML,
CveID: "CVE-2017-0002",
LastModified: new,
},
@@ -128,7 +128,7 @@ func TestIsCveInfoUpdated(t *testing.T) {
CveID: "CVE-2017-0003",
CveContents: models.NewCveContents(
models.CveContent{
Type: models.NVD,
Type: models.NvdXML,
CveID: "CVE-2017-0002",
LastModified: old,
},
@@ -150,7 +150,7 @@ func TestIsCveInfoUpdated(t *testing.T) {
CveID: "CVE-2017-0004",
CveContents: models.NewCveContents(
models.CveContent{
Type: models.NVD,
Type: models.NvdXML,
CveID: "CVE-2017-0002",
LastModified: old,
},
@@ -194,18 +194,18 @@ func TestDiff(t *testing.T) {
CveID: "CVE-2012-6702",
AffectedPackages: models.PackageStatuses{{Name: "libexpat1"}},
DistroAdvisories: []models.DistroAdvisory{},
CpeNames: []string{},
CpeURIs: []string{},
},
"CVE-2014-9761": {
CveID: "CVE-2014-9761",
AffectedPackages: models.PackageStatuses{{Name: "libc-bin"}},
DistroAdvisories: []models.DistroAdvisory{},
CpeNames: []string{},
CpeURIs: []string{},
},
},
Packages: models.Packages{},
Errors: []string{},
Optional: [][]interface{}{},
Optional: map[string]interface{}{},
},
},
inPrevious: models.ScanResults{
@@ -219,18 +219,18 @@ func TestDiff(t *testing.T) {
CveID: "CVE-2012-6702",
AffectedPackages: models.PackageStatuses{{Name: "libexpat1"}},
DistroAdvisories: []models.DistroAdvisory{},
CpeNames: []string{},
CpeURIs: []string{},
},
"CVE-2014-9761": {
CveID: "CVE-2014-9761",
AffectedPackages: models.PackageStatuses{{Name: "libc-bin"}},
DistroAdvisories: []models.DistroAdvisory{},
CpeNames: []string{},
CpeURIs: []string{},
},
},
Packages: models.Packages{},
Errors: []string{},
Optional: [][]interface{}{},
Optional: map[string]interface{}{},
},
},
out: models.ScanResult{
@@ -241,7 +241,7 @@ func TestDiff(t *testing.T) {
Packages: models.Packages{},
ScannedCves: models.VulnInfos{},
Errors: []string{},
Optional: [][]interface{}{},
Optional: map[string]interface{}{},
},
},
{
@@ -256,7 +256,7 @@ func TestDiff(t *testing.T) {
CveID: "CVE-2016-6662",
AffectedPackages: models.PackageStatuses{{Name: "mysql-libs"}},
DistroAdvisories: []models.DistroAdvisory{},
CpeNames: []string{},
CpeURIs: []string{},
},
},
Packages: models.Packages{
@@ -294,7 +294,7 @@ func TestDiff(t *testing.T) {
CveID: "CVE-2016-6662",
AffectedPackages: models.PackageStatuses{{Name: "mysql-libs"}},
DistroAdvisories: []models.DistroAdvisory{},
CpeNames: []string{},
CpeURIs: []string{},
},
},
Packages: models.Packages{
@@ -356,7 +356,7 @@ func TestIsCveFixed(t *testing.T) {
},
CveContents: models.NewCveContents(
models.CveContent{
Type: models.NVD,
Type: models.NvdXML,
CveID: "CVE-2016-6662",
LastModified: time.Time{},
},
@@ -374,7 +374,7 @@ func TestIsCveFixed(t *testing.T) {
},
CveContents: models.NewCveContents(
models.CveContent{
Type: models.NVD,
Type: models.NvdXML,
CveID: "CVE-2016-6662",
LastModified: time.Time{},
},
@@ -397,7 +397,7 @@ func TestIsCveFixed(t *testing.T) {
},
CveContents: models.NewCveContents(
models.CveContent{
Type: models.NVD,
Type: models.NvdXML,
CveID: "CVE-2016-6662",
LastModified: time.Time{},
},
@@ -415,7 +415,7 @@ func TestIsCveFixed(t *testing.T) {
},
CveContents: models.NewCveContents(
models.CveContent{
Type: models.NVD,
Type: models.NvdXML,
CveID: "CVE-2016-6662",
LastModified: time.Time{},
},