Merge branch 'release-v1.5.0'

.travis.yml

@@ -15,26 +15,37 @@ addons:
   apt:
     packages:
     - jq
+    - libxml2-utils 
 
 matrix:
   allow_failures:
     - jdk: oraclejdk9
 
 script:
   #run tests and integration tests
-  - mvn integration-test
-  - echo `mvn help:evaluate -Dexpression=project.version | grep -e '^[^\[]' | grep -v 'Downloading'` > iri.version
+  # see  https://stackoverflow.com/questions/34405047/how-do-you-merge-into-another-branch-using-travis-with-git-commands?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa
+  - build_head=$(git rev-parse HEAD) 
+  - git config --replace-all remote.origin.fetch +refs/heads/*:refs/remotes/origin/*
+  - git fetch origin dev
+  - git checkout -f dev
+  - git checkout $build_head
+  - git merge dev
+  - mvn integration-test -Dlogging-level=INFO
   #run jar sanity tests
-  - git clone https://github.com/alon-e/iri-regression
-  - cd iri-regression
+  - VERSION=`echo -e 'setns x=http://maven.apache.org/POM/4.0.0\ncat /x:project/x:version/text()' | xmllint --shell pom.xml | grep -v / | tr -d -` 
+  - echo $VERSION
+  - git clone https://github.com/iotaledger/iri-regression-tests.git
+  - cd iri-regression-tests
+  - git checkout -f master
+  - curl -LO https://s3.eu-central-1.amazonaws.com/iotaledger-dbfiles/dev/testnet_files.tgz
+  - tar -xzf testnet_files.tgz
   - mkdir iri
   - cp -rf ../target iri/target
-  - cp ../iri.version .
-  - bash run_all_stable_tests.sh `cat iri.version`
+  - bash run_all_stable_tests.sh $VERSION
   - cd ..
 
 after_success:
-  #codacy-coverage send report
+  #codacy-coverage send report. Uses Travis Env variable (CODACY_PROJECT_TOKEN)
   - test $TRAVIS_PULL_REQUEST = "false" && test $TRAVIS_JDK_VERSION = "oraclejdk8" && wget -O codacy-coverage-reporter-assembly-latest.jar $(curl https://api.github.com/repos/codacy/codacy-coverage-reporter/releases/latest | jq -r .assets[0].browser_download_url)
   - test $TRAVIS_PULL_REQUEST = "false" && test $TRAVIS_JDK_VERSION = "oraclejdk8" && java -jar codacy-coverage-reporter-assembly-latest.jar report -l Java -r target/site/jacoco/jacoco.xml
 
@@ -46,7 +57,7 @@ deploy:
   file: target/*.jar*
   skip_cleanup: true
   before_deploy: openssl aes-256-cbc -K $encrypted_5a15fa813cca_key -iv $encrypted_5a15fa813cca_iv -in codesigning.asc.enc -out codesigning.asc -d && gpg --fast-import codesigning.asc
-  deploy: mvn package -P build-extras
+  deploy: mvn package -P build-extras -Dlogging-level=INFO
   on:
     tags: true
     repo: iotaledger/iri

DOCKER.md

@@ -0,0 +1,74 @@
+## Quickstart
+
+Run the official iotaledger/iri container, passing the mandatory -p option:
+
+```docker run iotaledger/iri:v1.5.0 -p 14265```
+
+This will get your a running IRI with its API listening on port 14265, no neighbours and an empty database. The IRI Docker container by default expects data at /iri/data. Use the `-v` option of the `docker run` command to mount volumes so to have persistent data. You can also pass more command line options to the docker run command and those will be passed to IRI.
+
+If you want to use a iri.ini file with the docker container, supposing it's stored under /path/to/conf/iri.ini on your docker host, then pass `-v /path/to/conf:/iri/conf` and add -c /iri/conf/iri.ini as docker run arguments. So for example the `docker run` command above would become:
+
+```docker run -v /path/to/conf:/iri/conf -v /path/to/data:/iri/data iotaledger/iri:v1.5.0 -p 14265 -c /iri/conf/iri.ini```
+
+Please refer to the IRI documentation for further command line options and iri.ini options.
+
+## DOCKER and IRI in depth
+
+The Dockerfile included in this repo builds a working IRI docker container whilst trying to stay the least opinionated as possible. This allows system administrators the option to deploy and configure IRI based on their own individual circumstances and needs.
+
+When building IRI via the Dockerfile provided, Docker 17.05 minimum is required, due to the use of Docker build stages. During docker build, these are the stages invoked:
+- java: installs Oracle Java on top of Ubuntu
+- build: installs Maven on top of the java stage and compiles IRI
+- final container: copies the IRI jar file using the java stage as base
+
+The built container assumes the WORKDIR inside the container is /iri/data: this means that the database directory will be written inside that directory by default. If a system administrator wants to retain the database across restarts, it is his/her job to mount a docker volume in the right folder.
+
+The docker conatiner supports the env variables to configure advanced options. These variables can be set but are not required to run IRI.
+
+`JAVA_OPTIONS`: these are the java options to pass right after the java command. It must not contain -Xms nor -Xmx. Defaults to a safe value
+`JAVA_MIN_MEMORY`: the value of -Xms option. Defaults to 2G
+`JAVA_MAX_MEMORY`: the value of -Xmx option. Defaults to 4G
+`DOCKER_IRI_JAR_PATH`: defaults to /iri/target/iri*.jar as pushed by the Dockerfile. This is useful if custom IRI binaries want to be executed and the default path needs to be overridden
+`DOCKER_IRI_REMOTE_LIMIT_API`: defaults to "interruptAttachToTangle, attachToTangle, addNeighbors, removeNeighbors, getNeighbors"
+`DOCKER_IRI_MONITORING_API_PORT_ENABLE`: defaults to 0. If set to 1, a socat on port 14266 directed to 127.0.0.1:DOCKER_IRI_MONITORING_API_PORT_DESTINATION  will be open in order to allow all API calls regardless of the DOCKER_IRI_REMOTE_LIMIT_API setting. This is useful to give access to restricted API calls to local tools and still denying access to restricted API calls to the internet. It is highly recommended to use this option together with docker networks (docker run --net).
+
+The container entry point is a shell script that performs few additional steps before launching IRI:
+- verifies if `DOCKER_IRI_MONITORING_API_PORT_ENABLE` is set to 1
+- launches IRI with all parameters passed as desired
+
+It is important to note that other than --remote and --remote-limit-api "$DOCKER_IRI_REMOTE_LIMIT_API", neither the entrypoint nor the Dockerfile are aware of any IRI configuration option. This is to not tie the Dockerfile and its container to a specific set of IRI options. Instead, this contain still allows the use of an INI file or command line options. Please refer to the IRI documentation to learn what are the allowed options at command line and via the INI file.
+
+**At the time of writing, IRI requires -p to be passed either via INI or via command line. The entrypoint of this docker container does not do that for you.**
+
+Here is a systemd unit example you can use with this Docker container. This is just an example and customisation is possible and recommended. In this example the docker network iri must be created and the paths /mnt/iri/conf and /mnt/iri/data are used on the docker host to serve respectively the neighbors file and the data directory. No INI files are used in this example, instead options are passed via command line options, such as --testnet and --zmq-enabled.
+
+```
+[Unit]
+Description=IRI
+After=docker.service
+Requires=docker.service
+
+[Service]
+TimeoutStartSec=0
+Restart=always
+ExecStartPre=-/usr/bin/docker rm %n
+ExecStart=/usr/bin/docker run \
+--name %n \
+--hostname iri \
+--net=iri \
+-v /mnt/iri/conf:/iri/conf \
+-v /mnt/iri/data:/iri/data \
+-p 14265:14265 \
+-p 15600:15600 \
+-p 14600:14600/udp  \
+iotaledger/iri:v1.5.0 \
+-p 14265 \
+--zmq-enabled \
+--testnet
+
+ExecStop=/usr/bin/docker stop %n
+ExecReload=/usr/bin/docker restart %n
+
+[Install]
+WantedBy=multi-user.target
+```

Dockerfile

@@ -1,16 +1,84 @@
-FROM maven:3.5-jdk-8 as builder
+FROM ubuntu:18.04 as local_stage_java
+MAINTAINER [email protected]
+
+# Install Java
+ARG JAVA_VERSION=8u171-1
+RUN \
+  apt-get update && \
+  apt-get install -y software-properties-common --no-install-recommends && \
+  echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | debconf-set-selections && \
+  add-apt-repository -y ppa:webupd8team/java && \
+  apt-get update && \
+  apt-get install -y oracle-java8-installer=${JAVA_VERSION}~webupd8~0 --no-install-recommends && \
+  rm -rf /var/lib/apt/lists/* && \
+  rm -rf /var/cache/oracle-jdk8-installer
+
+# Define commonly used JAVA_HOME variable
+ENV JAVA_HOME /usr/lib/jvm/java-8-oracle
+
+# install maven on top of java stage
+FROM local_stage_java as local_stage_build
+ARG MAVEN_VERSION=3.5.3
+ARG USER_HOME_DIR="/root"
+ARG SHA=b52956373fab1dd4277926507ab189fb797b3bc51a2a267a193c931fffad8408
+ARG BASE_URL=https://apache.osuosl.org/maven/maven-3/${MAVEN_VERSION}/binaries
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+        curl \
+    && rm -rf /var/lib/apt/lists/*
+
+RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
+  && curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz \
+  && echo "${SHA}  /tmp/apache-maven.tar.gz" | sha256sum -c - \
+  && tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip-components=1 \
+  && rm -f /tmp/apache-maven.tar.gz \
+  && ln -s /usr/share/maven/bin/mvn /usr/bin/mvn
+
+ENV MAVEN_HOME /usr/share/maven
+ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"
+
+COPY docker/mvn-entrypoint.sh /usr/local/bin/mvn-entrypoint.sh
+COPY docker/settings-docker.xml /usr/share/maven/ref/
+
+VOLUME "$USER_HOME_DIR/.m2"
+
+# install build dependencies
+RUN apt-get update && apt-get install -y --no-install-recommends \
+        git \
+    && rm -rf /var/lib/apt/lists/*
+
 WORKDIR /iri
+
 COPY . /iri
 RUN mvn clean package
 
-FROM openjdk:jre-slim
-WORKDIR /iri
-COPY --from=builder /iri/target/iri-1.4.2.4.jar iri.jar
-COPY logback.xml /iri
-VOLUME /iri
+# execution image
+FROM local_stage_java
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+        jq curl socat \
+    && rm -rf /var/lib/apt/lists/*
+
+COPY --from=local_stage_build /iri/target/iri*.jar /iri/target/
+COPY docker/entrypoint.sh /
+
+# Java related options. Defaults set as below
+ENV JAVA_OPTIONS="-XX:+UnlockExperimentalVMOptions -XX:+DisableAttachMechanism -XX:InitiatingHeapOccupancyPercent=60 -XX:G1MaxNewSizePercent=75 -XX:MaxGCPauseMillis=10000 -XX:+UseG1GC"
+ENV JAVA_MIN_MEMORY 2G
+ENV JAVA_MAX_MEMORY 4G
+
+# Additional custom variables. See DOCKER.md for details
+ENV DOCKER_IRI_JAR_PATH "/iri/target/iri*.jar"
+ENV DOCKER_IRI_REMOTE_LIMIT_API "interruptAttachToTangle, attachToTangle, addNeighbors, removeNeighbors, getNeighbors"
 
-EXPOSE 14265
-EXPOSE 14777/udp
-EXPOSE 15777
+# Setting this to 1 will have socat exposing 14266 and pointing it on
+# localhost. See /entrypoint.sh
+# !!! DO NOT DOCKER EXPOSE (-p) 14266 as the remote api settings
+#     will not be applied on that port !!!
+# You also have to maintain $DOCKER_IRI_MONITORING_API_PORT_DESTINATION
+# based on the actual API port exposed via IRI
+ENV DOCKER_IRI_MONITORING_API_PORT_ENABLE 0
+ENV DOCKER_IRI_MONITORING_API_PORT_DESTINATION 14265
 
-CMD ["/usr/bin/java", "-XX:+DisableAttachMechanism", "-Xmx8g", "-Xms256m", "-Dlogback.configurationFile=/iri/conf/logback.xml", "-Djava.net.preferIPv4Stack=true", "-jar", "iri.jar", "-p", "14265", "-u", "14777", "-t", "15777", "--remote", "--remote-limit-api", "\"addNeighbors, removeNeighbors, getNeighbors\"", "$@"]
+WORKDIR /iri/data
+ENTRYPOINT [ "/entrypoint.sh" ]

README.md

@@ -6,26 +6,42 @@
 
 ## IOTA
 
-This is the main branch of the main IRI repository, as this is a IOTA reference implementation that is utilized specifically for what we have setup. It is a complete [[IOTA]](http://iota.org/) Node with a JSON-REST HTTP interface.
+The IRI repository is the main IOTA Reference Implementation and the embodiment of the IOTA network specification. 
 
-It allows to connect easily using java directly to a local or a remote [[IOTA node]](http://learn.iota.org/).
+This is a full-featured [[IOTA]](https://iota.org/) node with a convenient JSON-REST HTTP interface.
+It allows users to become part of the [[IOTA]](https://iota.org) network as both a transaction relay
+and network information provider through the easy-to-use [[API]](https://iota.readme.io/reference).
 
-* **Latest release:** 1.4.2.2 Release
-* **License:** GPLv3
+It is specially designed for users seeking a fast, efficient and fully-compatible network setup.
+
+Running an IRI node also allows light wallet users a node to directly connect to for their own wallet transactions.
+
+-* **License:** GPLv3
 
 # How to get started
 
-Obviously, because this is its own, independent network, you have to go through the same process as in the main network: **find neighbors**. You can find neighbors in the [[Discord Community]](https://discord.gg/7Gu2mG5), or on [[our forum]](https://forum.iota.org/). Community members are usually very happy to help you out and get you connected. If you want to get tokens for your testcase, please just ask in one of the communication channels as well.
+The IOTA network is an independent peer-to-peer network with a first-user, friend-to-friend, network structure:
 
-## Reporting Issues
+- As a 'first-user' network, to access the data streams and APIs that other users provide, you must first exchange your IP and port configuration with a current user.
 
-If you notice any issues or irregularities in this release. Please make sure to submit an issue on github.
+- As a 'friend-to-friend' network, you have the privilege of joining new users into the network through your node
+by adding them to your approved neighbors list — ensuring that you both broadcast to them and also receive their broadcasts.
 
+You can **find neighbors** quickly at both our [[Discord Community]](https://discord.gg/7Gu2mG5) and [[forum.iota.org]](https://forum.iota.org/).
+
+Everyone will be welcoming and very happy to help you get connected.
+If you want to get tokens for your testcase, please just ask in one of the communication channels.
 
-# Installing
+## Reporting Issues
 
-You have two options, the preferred option is that you compile yourself. The second option is that you utilize the provided jar, which is released regularly (when new updates occur) here: [Github Releases](https://github.com/iotaledger/iri/releases).
+If you notice any bugs, problems or other irregularities with this release,
+please submit an issue on github [[submit new issue]](https://github.com/iotaledger/iri/issues/new).
 
+# Installing
+
+The preferred option is that you compile yourself.
+The second option is that you utilize the provided jar, 
+which is released whenever there is a new update here: [Github Releases](https://github.com/iotaledger/iri/releases).
 
 ### Compiling yourself  
 
@@ -39,13 +55,22 @@ $ mvn clean compile
 $ mvn package
 ```
 
-This will create a `target` directory in which you will find the executable jar file that you can use for the 
+This will create a `target` directory in which you will find the executable jar file that you can use.
 
 ### How to run IRI 
 
 #### Locally
 
-Running IRI is pretty simple, and you don't even have to run it under admin rights. Below is a list of command line options. Here is an example script:
+Running IRI is quick and easy, and you can usually run it without admin rights.
+Below is a list of command line options.
+
+At a minimum, the port must be specified on the command-line — e.g., '`-p 14265`' 
+or in the `iota.ini` file — e.g., '`PORT = 14265`'.
+
+If the '`iota.ini`' file exists, it will be read.
+The port and all the command line options below take precedence over values specified in the ini config file.
+
+Here is an example script that specifies only the port, with all other setting to be read from the ini file **if it exists**:
 
 ```
 java -jar iri.jar -p 14265
@@ -63,29 +88,27 @@ command line arguments.
 
 Option | Shortened version | Description | Example Input
 --- | --- | --- | --- 
-`--port` | `-p` | This is a *mandatory* option that defines the port to be used to send API commands to your node | `-p 14800`
+`--port` | `-p` | This is a *mandatory* option that defines the port to be used to send API commands to your node | `-p 14265`
 `--neighbors` | `-n` | Neighbors that you are connected with will be added via this option. | `-n "udp://148.148.148.148:14265 udp://[2001:db8:a0b:12f0::1]:14265"`
 `--config` | `-c` | Config INI file that can be used instead of CLI options. See more below | `-c iri.ini`
-`--udp-receiver-port` | `-u` | UDP receiver port | `-u 14800`
-`--tcp-receiver-port` | `-t` | TCP receiver port | `-t 14800`
+`--udp-receiver-port` | `-u` | UDP receiver port | `-u 14600`
+`--tcp-receiver-port` | `-t` | TCP receiver port | `-t 15600`
 `--testnet` | | Makes it possible to run IRI with the IOTA testnet | `--testnet`
 `--remote` | | Remotely access your node and send API commands | `--remote`
 `--remote-auth` | | Require authentication password for accessing remotely. Requires a correct `username:hashedpassword` combination | `--remote-auth iotatoken:LL9EZFNCHZCMLJLVUBCKJSWKFEXNYRHHMYS9XQLUZRDEKUUDOCMBMRBWJEMEDDXSDPHIGQULENCRVEYMO`
 `--remote-limit-api` | | Exclude certain API calls from being able to be accessed remotely | `--remote-limit-api "attachToTangle, addNeighbors"`
 `--send-limit`| | Limit the outbound bandwidth consumption. Limit is set to mbit/s | `--send-limit 1.0`
 `--max-peers` | | Limit the number of max accepted peers. Default is set to 0 (mutual tethering) | `--max-peers 8`
-`--dns-resolution-false` | | Ignores DNS resolution refreshing  | --dns-resolution-false	
+`--dns-resolution-false` | | Ignores DNS resolution refreshing  | `--dns-resolution-false`	
 ### INI File
 
 You can also provide an ini file to store all of your command line options and easily update (especially neighbors) if needed. You can enable it via the `--config` flag. Here is an example INI file:
 ```
 [IRI]
-PORT = 14700
-UDP_RECEIVER_PORT = 14700
-NEIGHBORS = udp://my.favorite.com:15600
+PORT = 14265
+UDP_RECEIVER_PORT = 14600
+NEIGHBORS = udp://my.favorite.com:14600
 IXI_DIR = ixi
-HEADLESS = true
-DEBUG = true
+DEBUG = false
 DB_PATH = db
 ```
-

changelog.txt

@@ -1,3 +1,18 @@
+1.5.0
+    - Rework of the Tip Selection algorithm (#778)
+    - Validate the alpha value (#817)
+    - TipSelection: update API reference (#773)
+    - Inserted check for number of available processors. (#758)
+    - Improved Docker support (#744)
+    - Faster PearlDiver (PoW) (#733)
+    - Kerl hashing speed improvement (#628)
+    - Logging routing rework (#727)
+
+    Minor changes and fixes
+    - Fixed `attachmentTimestampUpperBound` value (#777)
+    - Fixed `getBalances` `tips` parameter parsing (#776)
+    - Added hash to `tx_trytes` ZMQ topic (#739)
+
 1.4.2.4
     - Tag indexing (#728)
     - TCP networking issues fixed (#592)

docker/entrypoint.sh

@@ -0,0 +1,15 @@
+#!/bin/bash
+# See Dockerfile and DOCKER.md for further info
+
+if [ "${DOCKER_IRI_MONITORING_API_PORT_ENABLE}" == "1" ]; then
+  nohup socat -lm TCP-LISTEN:14266,fork TCP:127.0.0.1:${DOCKER_IRI_MONITORING_API_PORT_DESTINATION} &
+fi
+
+exec java \
+  $JAVA_OPTIONS \
+  -Xms$JAVA_MIN_MEMORY \
+  -Xmx$JAVA_MAX_MEMORY \
+  -Djava.net.preferIPv4Stack=true \
+  -jar $DOCKER_IRI_JAR_PATH \
+  --remote --remote-limit-api "$DOCKER_IRI_REMOTE_LIMIT_API" \
+  "$@"