External Libraries & Templates

[JackFGS]

Good Afternoon,

Ive been struggling to setup github CI/CD for a couple of days now and I'm stuck!

Is there a way to include external libraries / templates to the container?

We use google drive and I have the drive mapped in the Linux GitHub runner but I cant see a way of adding it in the yaml?

Please see below any suggestions would be appreciated

name: example

on:
  push:
    paths:
    - '**.sch'
    - '**.kicad_pcb'
    - '**.yaml'
  pull_request:
    paths:
      - '**.sch'
      - '**.kicad_pcb'
      - '**.yaml'
  workflow_dispatch:
jobs:
  example:
    runs-on: [self-hosted, linux]
    steps:
    - uses: actions/checkout@v2
    - uses: INTI-CMNB/KiBot@v2_k8
      with:
        # Required - kibot config file
        config: config.kibot.yaml
        # optional - prefix to output defined in config
        dir: output
        # optional - schematic file
        schema: '*.kicad_sch'
        # optional - PCB design file
        board: '*.kicad_pcb'
    - name: upload results
      uses: actions/upload-artifact@v2
      with:
        name: output
        path: output

[set-soft]

I never used Google Drive for GitHub projects, what are you using to integrate it?

Normally you put your libs in another repo. Then you use git submodules. In this way the libs becomes part of the project, but you update them only in one repo. Note that you need to do some update in the repos using the lib, but is just a command, the changes are done in the lib repo.

This is the most common case.

[JackFGS]

I have finally managed to make it work today,

I have mounted google drive in the runner using https://github.com/astrada/google-drive-ocamlfuse and the yaml below!

 runs-on: [self-hosted, linux]
    container: 
      image: ghcr.io/inti-cmnb/kicad8_auto_full:latest   # Don't Change, except if needing older version of KiCad.
      options: -v /DRIVE:/src/Gdrive   
    steps:  
    - uses: actions/checkout@v4
    - name: Remove OUTPUT Dir
      run: |
        rm -rf ./OUTPUT
    - name: Run KiBot
      run: |
        kibot -c ./config.kibot.yaml
    - name: upload results
      uses: actions/upload-artifact@v4
      with:
       name: OUTPUT
       path: OUTPUT

[set-soft]

Interesting!
Are you installing the google-drive-ocamlfuse tool in a local runner?

[cyberreefguru]

To add to the question, how does one do the same thing when the user libraries are in a repo and you are using CI/CD? Hours of searching have turned up no answer.

Example, I have KICAD_USER_MODEL_DIR set to the location on my local drive when I'm using kicad manually -- how do I set this variable in kicad within docker when using CI?

I can pull the libraries into docker from git, but I don't know how to 'tell' kicad that is the user directory?

Specifically interested in:
KICAD8_3RD_PARTY
KICAD9_3RD_PARTY
KICAD_USER_MODEL_DIR
KICAD_USER_TEMPLATE_DIR

(though I don't think template dir is used).

I feel like this is trivial but I can't seem to find an answer (yet).

-Tom

[set-soft]

Did you try using the environment option from the global options?

[cyberreefguru]

I have not -- I will look into that. Thanks!

[cyberreefguru]

Hi Salvador -- I implemented global variables and also implemented some options for pcbdraw, but it's not clear they are working. I'm still getting warnings of missing libraries even though the libraries are available (based on listing the directory and echoing the output).

- 'Print schematic (PDF)' (print_sch) [pdf_sch_print]
- 'Top layer view' (board_top) [pcbdraw]
WARNING:(W103) (component) Component Resistor_SMD:R_0603_1608Metric has no footprint. (kibot - out_pcbdraw.py:45)
WARNING:(W103) (component) Component Capacitor_SMD:C_0603_1608Metric has no footprint. (kibot - out_pcbdraw.py:45)

These specific components are symlinks but that should not be a problem as far as I can tell -- they look fine when I pull the repo manually.

Snippet of my kibot.yaml:

  - name: board_top
    comment: "Top layer view"
    type: pcbdraw
    dir: doc
    options:
      output: power-module-pcb-top.png
      libs: 
        - pcbdraw/KiCAD-base
        - pcbdraw
      format: png
      margin: 5
      show_solderpaste: false
      show_components: all
      warnings: visible

and my git workflow:

    - name: Checkout PcbDraw Library
      uses: actions/checkout@v4
      with:
        repository: yaqwsx/PcbDraw-Lib
        path: pcbdraw

Or you can view here: https://github.com/crgmakes/module-power

Any suggestions?

I'd like to get this working but it's not a big deal if it doesn't.

Everything else is great, so kudos to you for the great work!

-Tom

[set-soft]

Hi @cyberreefguru !
You don't need to download pcbdraw libs, they are included with KiBot.
I tried your project and I don't have problems with resistors and capacitors ... but with everything else.

WARNING:(W103) (component) Component CRGM Connector:JST_SH_SM04B-SRSS-TB_1x04-1MP_P1.00mm_Horizontal has no footprint. (kibot - out_pcbdraw.py:45)
WARNING:(W103) (component) Component CRGM Symbols:HIGH-VOLTAGE-5MM has no footprint. (kibot - out_pcbdraw.py:45)
WARNING:(W103) (component) Component Converter_ACDC:Converter_ACDC_MeanWell_IRM-10-xx_THT has no footprint. (kibot - out_pcbdraw.py:45)
WARNING:(W103) (component) Component CRGM Power:SOTFL50P160X60-8N has no footprint. (kibot - out_pcbdraw.py:45)
WARNING:(W103) (component) Component Capacitor_SMD:CP_Elec_6.3x7.7 has no footprint. (kibot - out_pcbdraw.py:45)
WARNING:(W103) (component) Component CRGM Logos:Aquarius Logo 5mm has no footprint. (kibot - out_pcbdraw.py:45)
WARNING:(W103) (component) Component CRGM Passive:DFN1006-2 has no footprint. (kibot - out_pcbdraw.py:45)
WARNING:(W103) (component) Component CRGM Power:TO-252-2 has no footprint. (kibot - out_pcbdraw.py:45)
WARNING:(W103) (component) Component Inductor_SMD:L_Sunlord_MWSA0402S has no footprint. (kibot - out_pcbdraw.py:45)
WARNING:(W103) (component) Component CRGM Connector:GCT_USB4105-GF-A has no footprint. (kibot - out_pcbdraw.py:45)

Which is logic because PcbDraw doesn't have them.
In my experience, if you need a 2D view with all components, is much more simple to create a 3D view from top without perspective.

[cyberreefguru]

Yes, I am tracking. I pulled my library into the container but the path expansion was not working properly for some reason.

As you noted, I switched focused to the 3d model and figured out it was not using any of my models or the stock KiCad models - it was trying to download all 3rd party models - 80% of them failed. I turned off all downloads and managed to get my models imported and used, but now it can't find the stock models provided by KiCad for some reason:

WARNING:(W047) Missing 3D model for R3: ${KICAD9_3DMODEL_DIR}/Resistor_SMD.3dshapes/R_0603_1608Metric.wrl (kibot - out_base_3d.py:529)

I'm switching to a local container to take git out of the picture and debug more rapidly.

I will continue debugging.

[cyberreefguru]

Hi Salvador - I got docker running on my machine and launched your v9 image (kicad9_auto_full:latest). I cannot seem to find any 3d models or the 2d components.

root@c04d183b0cbb:~/.local/share/kicad/9.0# ls -AlF /usr/share/kicad
total 28
drwxr-xr-x 1 root root 4096 May 13 12:11 footprints/
drwxr-xr-x 3 root root 4096 May 13 12:07 plugins/
drwxr-xr-x 2 root root 4096 May 13 12:07 resources/
drwxr-xr-x 2 root root 4096 May 13 12:07 schemas/
drwxr-xr-x 1 root root 4096 May 13 12:07 scripting/
drwxr-xr-x 1 root root 4096 May 13 12:11 symbols/
drwxr-xr-x 1 root root 4096 May 13 12:07 template/

No 3dmodels directory. Local local directory:

root@c04d183b0cbb:~/.local/share/kicad/9.0# ls -AlF
total 32
drwxr-xr-x 2 root root 4096 May 21 22:51 3dmodels/
drwxr-xr-x 2 root root 4096 May 21 22:51 3rdparty/
drwxr-xr-x 2 root root 4096 May 21 22:51 footprints/
drwxr-xr-x 2 root root 4096 May 21 22:51 plugins/
drwxr-xr-x 2 root root 4096 May 21 22:51 projects/
drwxr-xr-x 3 root root 4096 May 21 22:51 scripting/
drwxr-xr-x 2 root root 4096 May 21 22:51 symbols/
drwxr-xr-x 2 root root 4096 May 21 22:51 template/

3dmodels directory, but it's empty:

root@ebd8d1eb9dff:~/.local/share/kicad/9.0# ls -AlF 3dmodels/
total 0
root@c04d183b0cbb:~/.local/share/kicad/9.0# ls -AlF 3rdparty/
total 0

I tried v8 auto_full:latest as well with the same results. Screen cap of the environment variables preference screen attached for both v8 and v9.

Paths are the same but no files. Not sure what I am missing?

-Tom

[cyberreefguru]

Ah, got it. I did end up loading the 3d models manually by pulling from gitlab, so I figured that out. Though for some reason I lost all the appearances and material details -- everything shows up grey, which it does not do when I run the program manually. One more thing to debug.

2d models are not working by default, I don't know why. I will work on that this weekend.

I don't see Fabrication toolkit included in the docker image -- is there an easy way to use that in Kibot?

[set-soft]

Ah, got it. I did end up loading the 3d models manually by pulling from gitlab, so I figured that out. Though for some reason I lost all the appearances and material details -- everything shows up grey, which it does not do when I run the program manually. One more thing to debug.

See #799 BTW: STEP files are for exact distance, not for beautiful renders, use VRML for this

2d models are not working by default, I don't know why. I will work on that this weekend.

Note that PcbDraw has 1% of the KiCad models, just the most commons, use 3D renders from top for components

I don't see Fabrication toolkit included in the docker image -- is there an easy way to use that in Kibot?

What has this that KiBot doesn't have?

[cyberreefguru]

1 - got it - I'll check that out.
2 - I get that, but it still can't find some of the basics, like Resistor_SMD:R_0603_1608Metric is there via a symlink
3 - all the stuff for JLC like part rotation and a properly formatted BOM for them so I don't have to hand edit things

[set-soft]

1 - got it - I'll check that out.
2 - I get that, but it still can't find some of the basics, like Resistor_SMD:R_0603_1608Metric is there via a symlink

I just tried using files in the repo for the docker image you mentioned:

src/kibot -b tests/board_samples/kicad_9/tests/board_samples/kicad_9/kibom-test-marked.kicad_pcb -c tests/yaml_samples/pcbdraw_all_components.kibot.yaml -d pp

This "board" has just components (not real board) and they get there.

3 - all the stuff for JLC like part rotation and a properly formatted BOM for them so I don't have to hand edit things

This is all done by KiBot, use the JLCPCB templates. No need for other tool.

[cyberreefguru]

Thanks for all the pointers. I will continue to work this.