Fixed issue in Migrate Module

Author: toolswatch

CHANGELOG.md

@@ -1,11 +1,15 @@
 Changelog
 =========
 
+0.7.2.1
+-----
+* [Fix] Imporved Migrate() module (SQLite to MongoDB). Thanks to Thiago Palmeira from Infolink for reporting the bug.
+
 0.7.2
 -----
-* Added support to CAPEC v2.10. Check [the full changelog](http://capec.mitre.org/data/reports/diff_reports/v2.9_v2.10.html).
-* Added support to CWE v2.11. Check [the full changelog](http://cwe.mitre.org/data/reports/diff_reports/v2.10_v2.11.html).
-* Added support to the new [Microsoft security update](https://portal.msrc.microsoft.com/en-us/security-guidance)
+* [New] Added support to CAPEC v2.10. Check [the full changelog](http://capec.mitre.org/data/reports/diff_reports/v2.9_v2.10.html).
+* [New] Added support to CWE v2.11. Check [the full changelog](http://cwe.mitre.org/data/reports/diff_reports/v2.10_v2.11.html).
+* [New] Added support to the new [Microsoft security update](https://portal.msrc.microsoft.com/en-us/security-guidance)
 * [Improve] Improved the `get_ms` method to returns both all and new Microsoft bulletins and KBs.
 * [Improve] Fixed issue #65. Cleaned the database from **Reject** entries.
 * [Doc] [Documentation](https://vfeed.io/docs) updated to reflect the new changes.
@@ -33,18 +37,18 @@ _All changes are immediate for consultancy / integrator license customers. The C
 
 0.6.9
 -----
-* The vFeed DB is no more available through `update` command. The command is deprecated. 
-* The delivery of the vFeed DB was handed over to a new established entity [vFeed IO](https://vfeed.io). This entity sets the goal to become the Leading Provider of Vulnerability and Threat Intelligence Database.
-* The API has been modified to reflect the new changes. 
+* [New] The vFeed DB is no more available through `update` command. The command is deprecated. 
+* [New] The delivery of the vFeed DB was handed over to a new established entity [vFeed IO](https://vfeed.io). This entity sets the goal to become the Leading Provider of Vulnerability and Threat Intelligence Database.
+* [New] The API has been modified to reflect the new changes. 
 
 0.6.8
 -----
-* Added support to CAPEC version 2.8. Check [about CAPEC v2.8](http://capec.mitre.org/news/index.html#december72015_CAPEC_List_Version_2.8_Now_Available).
-* Added support to CWE v2.9. Check [the full changelog](http://cwe.mitre.org/data/reports/diff_reports/v2.8_v2.9.html).
-* Added mapping to [WASC v2.0 Threat Classification](http://projects.webappsec.org/w/page/13246978/Threat%20Classification).
-* Added CVSS v2.0 vectors to `risk.py` class. Now, the methods `get_cvss` and `get_severity` display the vector when available.
-* Added new method `get_wasc` to reflect the new mapping with WASC v2.0. The method returns ID, Title and URL when available.
-* Modified the method `get_capec` to return the following:
+* [New] Added support to CAPEC version 2.8. Check [about CAPEC v2.8](http://capec.mitre.org/news/index.html#december72015_CAPEC_List_Version_2.8_Now_Available).
+* [New] Added support to CWE v2.9. Check [the full changelog](http://cwe.mitre.org/data/reports/diff_reports/v2.8_v2.9.html).
+* [New] Added mapping to [WASC v2.0 Threat Classification](http://projects.webappsec.org/w/page/13246978/Threat%20Classification).
+* [New] Added CVSS v2.0 vectors to `risk.py` class. Now, the methods `get_cvss` and `get_severity` display the vector when available.
+* [New] Added new method `get_wasc` to reflect the new mapping with WASC v2.0. The method returns ID, Title and URL when available.
+* [New] Modified the method `get_capec` to return the following:
     * The title
     * [Method of Attacks](http://capec.mitre.org/documents/schema/schema_v2.7.1.html#Method_of_Attack%20%28Methods_of_Attack%29)
     * [Mitigations](http://capec.mitre.org/documents/schema/schema_v2.7.1.html#Solution_or_Mitigation)
@@ -56,16 +60,16 @@ _All changes are immediate for consultancy / integrator license customers. The C
 
 0.6.7
 ---------
-* Added support to landscape.io with some code cleaning.
+* [New] Added support to landscape.io with some code cleaning.
 
 0.6.6
 ---------
-* Modified the `update.py` class to display the vFeed License before downloading the database.
+* [Improve] Modified the `update.py` class to display the vFeed License before downloading the database.
 
 0.6.5
 ---------
-* Added the ability to migrate to Mongo Database (Thanks so much to Ushan89 for the original code)
-* A new class `mongo.py` added (based on Ushan89 [code](https://github.com/ushan89/vFeed) to simply the process of migration from SQLite to MongoDB
+* [New] Added the ability to migrate to Mongo Database (Thanks so much to Ushan89 for the original code)
+* [New] A new class `mongo.py` added (based on Ushan89 [code](https://github.com/ushan89/vFeed) to simply the process of migration from SQLite to MongoDB
     * --migrate: Dump into a CSV then populate the vFeed MongoDB
 * The documentation updated. Visit [Documentation Page](https://github.com/toolswatch/vFeed/wiki/)
 

README.md

@@ -1,6 +1,6 @@
 vFeed The Correlated Vulnerability and Threat Intelligence Database Wrapper
 =======================================================================
-![vFeed](https://vfeed.io/wp-content/uploads/2016/07/vfeed.png)
+![vFeed](https://vfeed.io/wp-content/uploads/2016/08/logo_new-e1472658851686.png)
 
 [![Build Status](https://travis-ci.org/toolswatch/vFeed.svg?branch=master)](https://travis-ci.org/toolswatch/vFeed)
 [![Code Health](https://landscape.io/github/toolswatch/vFeed/master/landscape.svg?style=flat)](https://landscape.io/github/toolswatch/vFeed/master)
@@ -68,6 +68,10 @@ Refer to the [Documentation](https://vfeed.io/docs) official documentation page.
 Latest release
 ==============
 
+0.7.2.1
+-----
+* [Fix] Fixed Migrate() module (SQLite to MongoDB). Thanks to Thiago Palmeira from Infolink for reporting the bug.
+
 0.7.2
 -----
 * Added support to CAPEC v2.10. Check [the full changelog](http://capec.mitre.org/data/reports/diff_reports/v2.9_v2.10.html).

config/constants.py

@@ -13,11 +13,11 @@
 db_location = os.path.join(root_dir, db)
 
 # vFeed database information
-title = "vFeed - The Correlated Vulnerability and Threat Intelligence Database Wrapper"
+title = "vFeed - The Correlated Vulnerability and Threat Intelligence Database API"
 author = "vFeed IO"
 twitter = "@vfeed_io"
 repository = "https://vfeed.io"
-build = "0.7.1"
+build = "0.7.2.1"
 
 # Automated update Information
 dropbox_dl  = " INSERT YOUR DB LINK HERE"

config/stats.py

@@ -30,7 +30,7 @@ def get_stats(self):
             print("\t[-] Common Weakness Enumeration (CWE) types:", self.data[3])
             print("\t[-] Common Attack Pattern Enumeration and Classification (CAPEC) types:", self.data[4])
             print("\t[-] SecurityFocus BID:", self.data[5])
-            print("\t[-] OSVDB - Open Source Vulnerability Database advisories:", self.data[6])
+            print("\t[! DEPRECATED] OSVDB - Open Source Vulnerability Database advisories:", self.data[6])
             print("\t[-] CERT.org Vulnerability Notes:", self.data[7])
             print("\t[-] DOD-CERT Information Assurance Vulnerability Alert (IAVA):", self.data[8])
             print("\t[-] Scip AG Security Advisories:", self.data[9])
@@ -47,14 +47,13 @@ def get_stats(self):
             print("\t[-] HP (Hewlett Packard) Patches Advisories:", self.data[18])
             print("\t[-] Mandriva Patches Advisories:", self.data[19])
             print("\t[-] Microsoft Bulletins Advisories:", self.data[20])
-            print("\t[-] Microsoft KB Advisories:", self.data[21])
             print("\t[-] Redhat Patches Advisories:", self.data[22])
             print("\t[-] Redhat Bugzilla Advisories:", self.data[23])
 
             print("\n[+] Exploits and Proof of Concepts")
             print("\t[-] Exploit-DB Exploits:", self.data[24])
             print("\t[-] Metasploit Exploits /  Modules:", self.data[25])
-            print("\t[-] Milw0rm Exploits (Deprecated) :", self.data[26])
+            print("\t[! DEPRECATED] Milw0rm Exploits (Deprecated) :", self.data[26])
             print("\t[-] Saint Corporation Proof of Concepts and exploits:", self.data[27])
             print("\t[-] D2 Elliot Web Exploitation Framework:", self.data[34])
 

lib/migration/csvexports.sql

@@ -100,9 +100,6 @@ SELECT * FROM map_cve_msf;
 .once csv_exports/map_cve_ubuntu.csv
 SELECT * FROM map_cve_ubuntu;
 
-.once csv_exports/map_cve_mskb.csv
-SELECT * FROM map_cve_mskb;
-
 .once csv_exports/map_cve_vmware.csv
 SELECT * FROM map_cve_vmware;
 

lib/migration/mongo.py

@@ -7,15 +7,15 @@
 
 from __future__ import print_function
 
+import os
 import glob
 import subprocess
 
-from config.constants import migration_dir
+from config.constants import csv_dir
 from config.constants import mongo_conf
-from config.constants import migration_script
 from config.constants import db_location
-from config.constants import csv_dir
-
+from config.constants import migration_dir
+from config.constants import migration_script
 
 class Migrate(object):
     def __init__(self):
@@ -42,6 +42,7 @@ def do_sqlite_to_csv(self, ):
         """ read the vFeed.db and export entries to CSV
         :return: CSV files into csv_exports directory
         """
+        print("[+] Starting Migration Process ....")
         self.migration_read = '.read ' + self.migration_script
 
         try:
@@ -52,7 +53,7 @@ def do_sqlite_to_csv(self, ):
             ])
         except OSError as e:
             if e.errno == os.errno.ENOENT:
-                print('[Error] SQlite binary not found: install SQLite', e)
+                print("[Error] SQlite binary not found: install SQLite", e)
                 raise
             else:
                 raise
@@ -81,6 +82,6 @@ def do_csv_to_mongo(self, ):
                     '--headerline'
                 ])
             except Exception as e:
-                print('[Warning] Caught an exception', e)
+                print("[Warning] Caught an exception", e)
 
             print (("[+] Imported collection: {} --> vFeed MongoDB".format(self.table_name)))

tests.py

@@ -106,6 +106,11 @@
 print("Search for", txt)
 print(Search(txt).text())
 
+from config.stats import Stats
+
+Stats().get_stats()
+
 from lib.core.update import Update
 
 Update().update()
+

vfeedcli.py

@@ -23,7 +23,7 @@
 if __name__ == "__main__":
 
     parser = argparse.ArgumentParser()
-    parser.add_argument("-v", action="version", version=".:. {0} .:. ".format(title) + "API version: {0}".format(build))
+    parser.add_argument("-v", action="version", version="{0}".format(title) + " {0}".format(build))
     parser.add_argument("-m", "--method", metavar=('method', 'CVE'), help="Invoking multiple vFeed built-in functions",
                         nargs=2)
     parser.add_argument("-e", "--export", metavar=('json_dump', 'CVE'), help="Export the JSON content", nargs=2)
@@ -59,8 +59,9 @@
         # checking whether the MongoDB server is running
         # todo This test will be moved to Migrate class
         if mongo_server("mongod"):
-            print("[+] Mongo service is up. Starting migrating ....")
-            Migrate()
+            print("[+] Mongo service is up")
+            if Migrate():
+                print("[+] Migration successfully completed")
         else:
             print("[!] Mongo service is probably not up.")
 
@@ -87,3 +88,6 @@
         result = enum_classes(method_name, cve_id)
         if result is not False:
             print(result)
+
+    if len(sys.argv) < 2:
+        parser.print_help()