[FEATURE]Extract dtaas-gitlab-common Poetry package into lib/

[prasadtalasila]

Describe the Feature

As a DTaaS maintainer, I want to extract the shared GitLab client, token
handling, input validation, and idempotent resource operations into a
standalone dtaas-gitlab-common Poetry package at lib/ in the project root
so that both the Services CLI (deploy/services/cli/) and the DTaaS CLI
(cli/) depend on one implementation instead of maintaining duplicate GitLab
plumbing.

Problem Statement

The GitLab project-provisioning feature (#1681) is to be added to the DTaaS
CLI (cli/), which today has no GitLab API client at all — its
dependencies are click, tomlkit, python-on-whales, PyYAML,
cryptography. All existing GitLab client code lives in the Services CLI:

• deploy/services/cli/dtaas_services/pkg/services/gitlab/_api.py (~65 code
  lines) — SSL handling and the authenticated python-gitlab client factory.
• deploy/services/cli/dtaas_services/pkg/services/gitlab/personal_token.py
  (~179 code lines) — the token reader (~40 lines) is reusable; the
  OAuth-ROPC PAT creation is install-time work specific to the Services CLI.
• deploy/services/cli/dtaas_services/pkg/services/gitlab/validators.py
  (~73 code lines) — username, email, and password validation are all reusable
  now that the DTaaS CLI also sets GitLab user passwords (#1681 adds a
  password CSV column and a --password flag).
• deploy/services/cli/dtaas_services/pkg/services/gitlab/users.py
  (~194 code lines) — the idempotent user-creation logic
  (_create_single_user, _handle_gitlab_create_error: 409 → already-exists)
  is reusable now that the DTaaS CLI must also create GitLab users.

The DTaaS feature needs to create GitLab users, groups, and a project
structure under a chosen group (#1681). Without extraction, an estimated
~200+ lines of near-identical client/token/validation/idempotency code
would be duplicated across the two packages, and python-gitlab would become a
second, independently versioned dependency. Two copies drift; a fix to SSL
handling, token parsing, or idempotency would have to be made and reviewed
twice.

Proposed Solution

Create a publishable Poetry package that owns the provider-agnostic GitLab code
both CLIs share. URL resolution and secret creation stay in the consumers;
the shared primitives and idempotent resource operations move.

Package location and layout

lib/dtaas-gitlab-common/
├── pyproject.toml
├── README.md
├── dtaas_gitlab_common/
│   ├── __init__.py        # re-exports the public API
│   ├── client.py          # get_ssl_verify, get_gitlab_client
│   ├── tokens.py          # read_pat_from_json
│   ├── validators.py      # validate_username, validate_email
│   ├── retry.py           # call_with_backoff (satisfies #1681 retry need)
│   └── resources.py       # ensure_user, ensure_group, ensure_project
└── tests/
    ├── test_client.py
    ├── test_tokens.py
    ├── test_validators.py
    ├── test_retry.py
    └── test_resources.py

Public API surface

# client.py
def get_ssl_verify(default: bool = True) -> bool: ...
def get_gitlab_client(
    base_url: str, private_token: str, ssl_verify: bool = True
) -> gitlab.Gitlab: ...

# tokens.py
def read_pat_from_json(
    path: str | Path, key: str = "personal_access_token"
) -> tuple[bool, str]: ...   # (success, token_or_error)

# validators.py
def validate_username(username: str) -> str | None: ...   # error msg or None
def validate_email(email: str) -> str | None: ...
def validate_password(password: str) -> str | None: ...

# retry.py
def call_with_backoff(fn, *, retries: int = 3, base_delay: float = 0.5): ...

# resources.py  (idempotent — probe/create, safe to re-run)
def ensure_user(gl, payload: dict) -> tuple[bool, str, int | None]: ...
def ensure_group(gl, full_path: str, **attrs) -> tuple[bool, str, int | None]: ...
def ensure_project(
    gl, name: str, namespace: str, **attrs
) -> tuple[bool, str, int | None]: ...

What moves, and what deliberately does not

Source (Services CLI)	Destination	Notes
_api.py → get_ssl_verify, get_gitlab_client	client.py	Factory takes an explicit base_url; does not read env.
_api.py → build_base_url, _validate_gitlab_port, _validate_hostname	stays in Services CLI	Reads GITLAB_PORT/HOSTNAME; that URL policy is Services-CLI specific.
personal_token.py → token reader	tokens.py as read_pat_from_json(path, key)	Path/key parameterised.
personal_token.py → OAuth-ROPC PAT creation	stays in Services CLI	Install-time concern; not needed by DTaaS.
validators.py → _validate_username, _validate_email, _validate_password	validators.py	Field-level validators, all shared (DTaaS now sets passwords too).
validators.py → validate_user_row	stays in Services CLI	Row composition differs per package (Services has no groups/load_balance).
users.py → _create_single_user, _handle_gitlab_create_error	resources.py as ensure_user	Idempotent user creation, now shared by both CLIs.
users.py → CSV/token orchestration (setup_gitlab_users)	stays in Services CLI	Package-specific workflow.

ensure_group and ensure_project are new idempotent helpers (no prior
implementation) required by the DTaaS feature and placed in common so the
project structure logic is not DTaaS-only.

pyproject.toml (new package)

[tool.poetry]
name = "dtaas-gitlab-common"
version = "0.1.0"
description = "Shared GitLab client, token, validation, and resource helpers for DTaaS CLIs"
authors = ["The INTO-CPS-Association"]
packages = [{ include = "dtaas_gitlab_common" }]

[tool.poetry.dependencies]
python = "^3.10"
python-gitlab = "^5.0"      # pin to the version the Services CLI resolves today
urllib3 = "*"

[tool.poetry.group.dev.dependencies]
pytest = "*"
pytest-cov = "*"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Consumption model

• In-repo development: each CLI adds a Poetry path dependency to
  lib/dtaas-gitlab-common with develop = true (relative path per package —
  ../lib/... from cli/, ../../../lib/... from deploy/services/cli/).
• Release: publish to PyPI and pin dtaas-gitlab-common = "^0.1.0" in both.

Alternatives Considered

• Copy the code into the DTaaS CLI. Cheapest immediately, but locks in the
  duplication this issue exists to prevent and doubles maintenance/review.
• Import dtaas-services as a library from the DTaaS CLI. Reuses the client
  without a new package, but couples the application-layer CLI to the infra
  package's release cadence and its bundled-GitLab assumptions (env URL, root
  token) — cutting against placing #1681 in the DTaaS CLI, which must also
  serve external-GitLab deployments.
• Share source across the monorepo without packaging. Breaks independent
  versioning/publishing of the two CLIs, which ship as separate pip packages.

Additional Context

Prerequisite for 02-migrate-dtaas-services.md (Services CLI refactors onto the
package — done first) and 03-add-gitlab-provisioning-dtaas.md (DTaaS CLI adds
#1681 on top of the package and #1690). The Services CLI already depends on
python-gitlab, so the shared dependency is not new to that package.

Success Criterion

Describe the expected outcome, using a checklist where appropriate.

Checklist:

• dtaas-gitlab-common exists at lib/dtaas-gitlab-common/ and builds with
  poetry build.
• Public API (get_gitlab_client, get_ssl_verify, read_pat_from_json,
  validate_username, validate_email, validate_password,
  call_with_backoff, ensure_user, ensure_group, ensure_project) is
  exported from dtaas_gitlab_common/__init__.py.
• get_gitlab_client takes an explicit base_url and does not read
  environment variables.
• read_pat_from_json is path- and key-parameterised (no hard-coded
  filename or base dir).
• ensure_user/ensure_group/ensure_project are idempotent (probe →
  create; existing → no-op) and return a (success, message, id) contract.
• call_with_backoff retries only transient failures (timeouts, 5xx) and
  re-raises 4xx immediately.
• Test coverage is improved — unit tests for client construction (SSL on/
  off), token reading, validators, retry, and each idempotent resource helper
  (create, already-exists, duplicate), ported from the existing Services CLI
  GitLab tests.
• No qlty issues.