Skip to content

[Feature] dynamic number of instances#22

Open
sofianinho wants to merge 13 commits intoeucuepo:masterfrom
sofianinho:master
Open

[Feature] dynamic number of instances#22
sofianinho wants to merge 13 commits intoeucuepo:masterfrom
sofianinho:master

Conversation

@sofianinho
Copy link
Copy Markdown

Hey Eugenio,
I propose this pull request with this list of new features:

  • The number of Zookeeper and Broker virtual machines is now dynamic and set by the user
    • The default value is 2. There is no predefined values, or limit.
    • To change the value, set the VAGRANT_ZKS and VAGRANT_BRS value in the environment before vagrant up command.
  • The size of the VM is also dynamic and can be set also through environment variables
    • The dafault value is 2 vCPUs and 2048MB of RAM for both VMs
    • These are the environment variables you want to change if you want to tweak that: VAGRANT_ZK_CPU, VAGRANT_ZK_RAM for zookeeper and VAGRANT_BR_CPU, VAGRANT_BR_RAM for the broker.
  • The subnet for machines is also configurable
    • The default is 10.192.133.0 and can be set through VAGRANT_SUBNET in the environment
    • Additional useful network parameters that are configurable are: VAGRANT_GW(default 10.192.133.1) and VAGRANT_EXTERNAL_IF (default eno4).
    • The values of the network parameters are tailored to my environment. Your host is probably different. Set the default gateway and outgoing interface that matches your own host to allow the machines to connect to the Internet and to be reachable.
  • The VMs guest operating system is centos/8 64-bit
  • The kafka version is now 2.5.0
  • The user scripts in the original project are updated to make requests to a dynamic number of brokers.
  • There is a CPU limit artificially added to the VMs to tell the hypervisor not to overload the host's CPU if the VM is running at full power. This is a percentage, and sets the cpuexecutioncap of the hypervisor through the VAGRANT_CPU_LIMIT variable (default 50).
  • The 4lw.commands.whitelist are on by default now. See more on the use below.
  • I recommand maintainig a kafkarc file with all your environment variables as those are volatile in your shell. Source this file each time you log in your host, or change terminal shell so that vagrant can get those values from the environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant