Skip to content

Issues during installation in Ubuntu #79

@lllilllilllilili

Description

@lllilllilllilili

Hello (VM : Ubuntu 16.04LTS)

I'd like to ask you a few questions.

About fluent-plugin-gcloud-pubsub-custom Installation Issues ( If I have any problems registering Issue, please let me know )

I was practicing the configuration of installing fluid on ubuntu and sending it to pubsub. (ruby using /.rvm/gems/ruby-2.5.8)

Write the instructions in accordance with the guidelines provided by github.

I ran

$ sudo /etc/init.d/td-agent start

Failed.

[....] Starting td-agent (via systemctl): td-agent.serviceJob for td-agent.service failed because the control process exited with error code. See "systemctl status td-agent.service" and "journalctl -xe" for details.
failed!

So I looked at the failure, and I couldn't check the plugin.

config error file="/etc/td-agent/td-agent.conf" error="Unknown output plugin 'gcloud_pubsub'. Run 'gem search -rd fluent-plugin'

/etc/td-agent/td-agent.conf

Page up
<source>
  @type tail
  path /var/log/nginx/access.log
  pos_file /var/log/td-agent/nginx-access.pos
  tag example.publish
  format nginx
</source>
#<match input.twitter>
#   @type stdout
#</match>
<match example.publish>
  type gcloud_pubsub
  project ...
  key ...
  topic projects...
  autocreate_topic false
  max_messages 1000
  max_total_size 9800000
  max_message_size 4000000
  timeout 60
  <buffer>
    @type memory
    flush_interval 1s
  </buffer>
  <format>
    @type json
  </format>
</match>

** project, topic, key는 ... I've done with the secret.

I checked if the plug-in was installed, but it wasn't either.

$ sudo td-agent-gem list | egrep "pubsub|google"

/var/log/td-agent/td-agent.log

2021-08-03 07:00:01 +0000 [info]: starting fluentd-0.12.40
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-mixin-plaintextformatter' version '0.2.6'
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-plugin-kafka' version '0.6.1'
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-plugin-mongo' version '0.8.1'
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.5.6'
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-plugin-s3' version '0.8.5'
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-plugin-scribe' version '0.10.14'
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-plugin-td' version '0.10.29'
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-plugin-td-monitoring' version '0.2.3'
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-plugin-webhdfs' version '0.7.1'
2021-08-03 07:00:01 +0000 [info]: gem 'fluentd' version '0.12.40'
2021-08-03 07:00:01 +0000 [info]: adding match pattern="td.*.*" type="tdlog"
2021-08-03 07:00:01 +0000 [info]: adding match pattern="debug.**" type="stdout"
2021-08-03 07:00:01 +0000 [info]: adding source type="forward"
2021-08-03 07:00:01 +0000 [info]: adding source type="http"
2021-08-03 07:00:01 +0000 [info]: adding source type="debug_agent"
2021-08-03 07:00:01 +0000 [info]: using configuration file: <ROOT>
  <match td.*.*>
    @type tdlog
    apikey xxxxxx
    auto_create_table
    buffer_type file
    buffer_path /var/log/td-agent/buffer/td
    buffer_chunk_limit 33554432
    <secondary>
      @type file
      path /var/log/td-agent/failed_records
      buffer_path /var/log/td-agent/failed_records.*
    </secondary>
  </match>
  <match debug.**>
    @type stdout
  </match>
  <source>
    @type forward
  </source>
  <source>
    @type http
    port 8888
  </source>
  <source>
    @type debug_agent
    bind 127.0.0.1
    port 24230
  </source>
</ROOT>
2021-08-03 07:00:01 +0000 [info]: listening fluent socket on 0.0.0.0:24224
2021-08-03 07:00:01 +0000 [info]: listening dRuby uri="druby://127.0.0.1:24230" object="Engine"
2021-08-03 07:23:37 +0000 [info]: shutting down fluentd
2021-08-03 07:23:37 +0000 [info]: shutting down input type="forward" plugin_id="object:3feb28faeb34"
2021-08-03 07:23:37 +0000 [info]: shutting down input type="http" plugin_id="object:3feb28fa9f08"
2021-08-03 07:23:37 +0000 [info]: shutting down input type="debug_agent" plugin_id="object:3feb28fa8f2c"
2021-08-03 07:23:37 +0000 [info]: shutting down output type="tdlog" plugin_id="object:3feb28d7b40c"
2021-08-03 07:23:37 +0000 [info]: shutting down output type="stdout" plugin_id="object:3feb28ff6e70"
2021-08-03 07:23:37 +0000 [info]: process finished code=0
2021-08-03 07:23:38 +0000 [info]: reading config file path="/etc/td-agent/td-agent.conf"
2021-08-03 07:23:38 +0000 [info]: starting fluentd-0.12.40
2021-08-03 07:23:38 +0000 [info]: gem 'fluent-mixin-plaintextformatter' version '0.2.6'
2021-08-03 07:23:38 +0000 [info]: gem 'fluent-plugin-buffer-lightening' version '0.0.2'
2021-08-03 07:23:38 +0000 [info]: gem 'fluent-plugin-kafka' version '0.6.1'
2021-08-03 07:23:38 +0000 [info]: gem 'fluent-plugin-mongo' version '0.8.1'
2021-08-03 07:23:38 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.5.6'
2021-08-03 07:23:38 +0000 [info]: gem 'fluent-plugin-s3' version '0.8.5'
2021-08-03 07:23:38 +0000 [info]: gem 'fluent-plugin-scribe' version '0.10.14'
2021-08-03 07:23:38 +0000 [info]: gem 'fluent-plugin-td' version '0.10.29'

I'm really curious about the reason why it can't be installed..

Thank you

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions