Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,15 @@
# Changelog

All notable changes to Data Hopper EDW (formerly Data Hopper EDW) are documented in this file.
All notable changes to Data Hopper EDW (formerly hop-datavault) are documented in this file.

## Unreleased

### Maven coordinates and install path

- Maven: `org.hopper:hopper-edw` (was `org.apache.hop:hop-datavault`; the rebrand briefly used `org.projectdatahopper.hop`)
- Plugin zip unpacks to `plugins/misc/hopper-edw/` (was `plugins/misc/datavault/`). Remove the old folder before installing this build
- Thin JDBC module groupId is `org.hopper` as well (`hop-hsm-jdbc`)

### Java packages `org.hopper.edw`

- Plugin code lives under `org.hopper.edw.datavault`, `org.hopper.edw.catalog`, `org.hopper.edw.quality`, `org.hopper.edw.databases.hopsourcemodel`, and `org.hopper.edw.hsm.jdbc` (was `org.apache.hop.*` for those trees)
Expand All @@ -12,10 +18,10 @@ All notable changes to Data Hopper EDW (formerly Data Hopper EDW) are documented

### Product identity: Data Hopper EDW

- Repository home: [ProjectDataHopper/hopper-edw](https://github.com/ProjectDataHopper/hopper-edw) (transferred from `ProjectDataHopper/hopper-edw`; old GitHub URLs redirect)
- Repository home: [ProjectDataHopper/hopper-edw](https://github.com/ProjectDataHopper/hopper-edw) (transferred from `mattcasters/hop-data-vault`; old GitHub URLs redirect)
- Product name: **Data Hopper EDW** — Apache Hop plugins to build an Enterprise Data Warehouse, not a Data Vault-only tool
- Maven coordinates: `org.projectdatahopper.hop:hopper-edw` (was `org.apache.hop:hop-datavault`). Marketplace: `./hop marketplace install hopper-edw`
- Assembly zip: `target/hopper-edw-*.zip`. Unzip layout stays `plugins/misc/datavault/`
- Maven coordinates: `org.hopper:hopper-edw`. Marketplace: `./hop marketplace install hopper-edw`
- Assembly zip: `target/hopper-edw-*.zip`. Unzip layout: `plugins/misc/hopper-edw/`
- License remains Apache License 2.0; copyright **i-Bridge bv**. Not an Apache Software Foundation release
- **0.9.0 cutover:** uninstall `$HOP_HOME/plugins/misc/datavault` from the old `hop-datavault` artifact, re-import [`hop-marketplace-repo.yaml`](hop-marketplace-repo.yaml), then install `hopper-edw`. Release 0.9.0 remains at `org.apache.hop:hop-datavault` on Nexus

Expand Down
11 changes: 6 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,12 @@ This file is for **AI coding assistants** and third-party contributors. Read it

**Data Hopper EDW** (`hopper-edw`) is a set of Apache Hop plugins to build an **Enterprise Data Warehouse**: visual models (`.hsm` / `.hdv` / `.hbv` / `.hdm`), catalog-first sources (including multi-table composite feeds), model-driven DDL and load pipelines, lineage, schema validation gates, data quality, execution maps (`.hem`), Hop Lineage Views (`.hlv`), the **EDW Journey** perspective, and optional AI help.

- Maven artifact: `org.projectdatahopper.hop:hopper-edw` (see `pom.xml` for current version)
- Install layout: `plugins/misc/datavault/` (from the assembly zip under `target/`)
- Maven artifact: `org.hopper:hopper-edw` (see `pom.xml` for current version)
- Install layout: `plugins/misc/hopper-edw/` (from the assembly zip under `target/`)
- Java packages: `org.hopper.edw.datavault`, `.catalog`, `.quality`, `.databases.hopsourcemodel`, `.hsm.jdbc`
- Upstream repo: https://github.com/ProjectDataHopper/hopper-edw
- Legal: Apache License 2.0, copyright i-Bridge bv. Not an ASF release.
- Do **not** rename Java packages, plugin IDs, file extensions, metadata keys, or `jdbc:hop-hsm:`.
- Do **not** rename plugin IDs, file extensions, metadata keys, or `jdbc:hop-hsm:`.

## Hard version pins

Expand Down Expand Up @@ -110,7 +111,7 @@ Do **not** “fix” Hop core inside this repo. Core Hop changes belong in **apa
Requirements: **JDK 21**, **Maven 3.x**.

```bash
mvn clean package # unit tests + jar + plugin zip (includes plugins/misc/datavault/docs/)
mvn clean package # unit tests + jar + plugin zip (includes plugins/misc/hopper-edw/docs/)
mvn test # unit tests only
mvn generate-resources # refresh docs/images/diagrams/*.svg from docs/diagrams/*.puml
mvn spotless:apply # format before commit (Google Java Format)
Expand All @@ -120,7 +121,7 @@ mvn spotless:check # CI-style format check
Artifacts:

- `target/hopper-edw-*-SNAPSHOT.jar`
- `target/hopper-edw-*-SNAPSHOT.zip` — unzip into `$HOP_HOME` (layout: `plugins/misc/datavault/`)
- `target/hopper-edw-*-SNAPSHOT.zip` — unzip into `$HOP_HOME` (layout: `plugins/misc/hopper-edw/`)

Notes:

Expand Down
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ Highlights:

Full index: **[docs/README.md](docs/README.md)**

`mvn package` also builds an HTML copy of the user guide into `target/generated-docs/` and ships it in the plugin zip at `plugins/misc/datavault/docs/index.html`. Architecture diagrams are committed SVGs generated from PlantUML (`docs/diagrams/`).
`mvn package` also builds an HTML copy of the user guide into `target/generated-docs/` and ships it in the plugin zip at `plugins/misc/hopper-edw/docs/index.html`. Architecture diagrams are committed SVGs generated from PlantUML (`docs/diagrams/`).

| Audience | Document |
|----------|----------|
Expand Down Expand Up @@ -134,7 +134,7 @@ Artifacts:
- `target/hopper-edw-0.10.0-SNAPSHOT.jar`
- `target/hopper-edw-0.10.0-SNAPSHOT.zip` (ready-to-unzip plugin layout)

Maven coordinates from **0.10.0** onward: `org.projectdatahopper.hop:hopper-edw`. Release **0.9.0** was published as `org.apache.hop:hop-datavault`.
Maven coordinates from **0.10.0** onward: `org.hopper:hopper-edw`. Release **0.9.0** was published as `org.apache.hop:hop-datavault` and unpacked under `plugins/misc/datavault/`.

Published release artifacts for **0.9.0** (previous coordinates):

Expand All @@ -144,7 +144,7 @@ Published release artifacts for **0.9.0** (previous coordinates):
## Installation (external plugin)

1. Install **Apache Hop 2.19.0**.
2. Unzip the assembly zip into your Hop installation root (`$HOP_HOME`) so both `plugins/misc/datavault/` and `lib/core/` (Jinjava) are updated. Copying only the plugin jar is not enough for Jinja SQL.
2. Unzip the assembly zip into your Hop installation root (`$HOP_HOME`) so both `plugins/misc/hopper-edw/` and `lib/core/` (Jinjava) are updated. Copying only the plugin jar is not enough for Jinja SQL.
3. Restart Hop GUI.
4. New metadata types appear under **Metadata → Data Vault**. **Data Vault Update**, **Business Vault Update**, **Validate resource definitions**, **Export data lineage**, and **Update resource definition group** actions are available in workflows. `.hsm`, `.hdv`, `.hbv`, `.hdm`, and `.hlv` files open in Explorer tabs.

Expand All @@ -163,7 +163,7 @@ Continuous Jenkins builds publish the latest SNAPSHOT zip to the Data Hopper com
Updated repository 'data-hopper-community' → https://repository.data-hopper.com/repository/hop-community-plugins/ (browse enabled)
```

If you previously installed **0.9.0** as `hop-datavault`, remove `$HOP_HOME/plugins/misc/datavault` first so the folders do not stack.
If you previously installed **0.9.0** as `hop-datavault`, remove `$HOP_HOME/plugins/misc/datavault` first so the old and new folders do not stack.

**2. Query** available plugins (filter for this one):

Expand All @@ -182,10 +182,10 @@ If you previously installed **0.9.0** as `hop-datavault`, remove `$HOP_HOME/plug
```

```text
Resolved hopper-edw → org.projectdatahopper.hop:hopper-edw:0.10.0-SNAPSHOT (prefer repo 'data-hopper-community')
… Marketplace - Downloading org.projectdatahopper.hop:hopper-edw:0.10.0-SNAPSHOT from https://repository.data-hopper.com/repository/hop-community-plugins/…
… Marketplace - Installed org.projectdatahopper.hop:hopper-edw:0.10.0-SNAPSHOT. Restart Hop to load the plugin.
Plugin org.projectdatahopper.hop:hopper-edw:0.10.0-SNAPSHOT installed under $HOP_HOME from repo 'data-hopper-community'. Restart Hop to load it.
Resolved hopper-edw → org.hopper:hopper-edw:0.10.0-SNAPSHOT (prefer repo 'data-hopper-community')
… Marketplace - Downloading org.hopper:hopper-edw:0.10.0-SNAPSHOT from https://repository.data-hopper.com/repository/hop-community-plugins/…
… Marketplace - Installed org.hopper:hopper-edw:0.10.0-SNAPSHOT. Restart Hop to load the plugin.
Plugin org.hopper:hopper-edw:0.10.0-SNAPSHOT installed under $HOP_HOME from repo 'data-hopper-community'. Restart Hop to load it.
```

You can also use **Tools → Marketplace…** in Hop GUI: import the repository on the **Repositories** tab, then install from the **Plugins** tab.
Expand Down
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Documentation for the **Data Hopper EDW** plugin (development **0.10.0-SNAPSHOT*

Architecture pictures are **committed SVGs** (`docs/images/diagrams/`) generated from PlantUML (`docs/diagrams/`). Do not put Mermaid or Markdown ` ``` ` fences in `.adoc` files.

`mvn package` writes an HTML copy of this guide to `target/generated-docs/` and into the plugin zip at `plugins/misc/datavault/docs/` (open `index.html`).
`mvn package` writes an HTML copy of this guide to `target/generated-docs/` and into the plugin zip at `plugins/misc/hopper-edw/docs/` (open `index.html`).

## Start here

Expand Down
2 changes: 1 addition & 1 deletion docs/hop-web-modelers.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ Painters already target `IGc`. Interactive web path uses the `*ModelCanvasSvgRen

1. Install Hop with the canvas SPI and current web client scripts (`core` / `engine` / `ui` / `rap` as needed).
2. Build this plugin: `mvn clean package -DskipTests`
3. Unzip `target/hopper-edw-*-SNAPSHOT.zip` into the Hop Web home (not only `plugins/misc/datavault/`) so Jinjava jars land in `lib/core`.
3. Unzip `target/hopper-edw-*-SNAPSHOT.zip` into the Hop Web home (not only `plugins/misc/hopper-edw/`) so Jinjava jars land in `lib/core`.
4. Start Hop Web (for example `docker/run-hop-web-local.sh --quick` from a Hop checkout that mounts the plugin).
5. Create or open a model under the explorer.

Expand Down
4 changes: 2 additions & 2 deletions docs/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

image::images/edw-logo.svg[EDW,width=64,height=64]

Offline HTML set generated by `mvn package` and shipped in the plugin zip under `plugins/misc/datavault/docs/`. In Hop GUI, open it from the main toolbar **EDW** button or **Help → EDW documentation**.
Offline HTML set generated by `mvn package` and shipped in the plugin zip under `plugins/misc/hopper-edw/docs/`. In Hop GUI, open it from the main toolbar **EDW** button or **Help → EDW documentation**.

Sources are the AsciiDoc files in the repository `docs/` folder. Architecture pictures are committed SVGs (PlantUML sources in link:diagrams/README.adoc[docs/diagrams/]).

Expand Down Expand Up @@ -126,4 +126,4 @@ From the repository root (JDK 21, Maven 3):
mvn package -DskipTests
----

Then open `target/generated-docs/index.html`, or unzip `target/hopper-edw-*-SNAPSHOT.zip` and open `plugins/misc/datavault/docs/index.html`.
Then open `target/generated-docs/index.html`, or unzip `target/hopper-edw-*-SNAPSHOT.zip` and open `plugins/misc/hopper-edw/docs/index.html`.
2 changes: 1 addition & 1 deletion docs/plans/jinja-dbt-import-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@ Jinjava has a history of sandbox escapes. Basics must:
- Restrict EL to registered functions + Jinja builtins; no `getClass`, no interpreter exposure.
- Unit-test that `{{ ''.class }}` / Java method probes fail.

Bundle Jinjava + required transitives (Guava, JSoup, shaded JUEL, etc. as needed) in `src/assembly/assembly.xml` under `plugins/misc/datavault/lib`. Attribute Apache-2.0 in `NOTICE`.
Bundle Jinjava + required transitives (Guava, JSoup, shaded JUEL, etc. as needed) in `src/assembly/assembly.xml` under `plugins/misc/hopper-edw/lib`. Attribute Apache-2.0 in `NOTICE`.

### 4.4 YAML

Expand Down
2 changes: 1 addition & 1 deletion docs/source-modeler-overview.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ With Data Hopper EDW installed, create a connection of type **Apache Hop Source
| Username / password | Hop Server credentials |
|===

The driver class is `org.hopper.edw.hsm.jdbc.HopHsmJdbcDriver`. The thin client jar is shipped under `plugins/misc/datavault/lib/` so **Table input**, Explore database, and other Hop database consumers can use the connection without a separate driver install.
The driver class is `org.hopper.edw.hsm.jdbc.HopHsmJdbcDriver`. The thin client jar is shipped under `plugins/misc/hopper-edw/lib/` so **Table input**, Explore database, and other Hop database consumers can use the connection without a separate driver install.

===== 1. Create a Source model service (Hop)

Expand Down
2 changes: 1 addition & 1 deletion hop-hsm-jdbc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ With the **Data Hopper EDW** plugin installed, create a relational connection of
| Database name | Optional default **Source model service** (JDBC schema) |
| Username / password | Hop Server credentials |

JDBC URL shape: `jdbc:hop-hsm://{host}:{port}/{service}`. Extra options use `?` / `&` (e.g. `rowLimit`, `connectTimeout`, `readTimeout`). The driver jar is bundled under `plugins/misc/datavault/lib/`.
JDBC URL shape: `jdbc:hop-hsm://{host}:{port}/{service}`. Extra options use `?` / `&` (e.g. `rowLimit`, `connectTimeout`, `readTimeout`). The driver jar is bundled under `plugins/misc/hopper-edw/lib/`.

## DBeaver setup

Expand Down
2 changes: 1 addition & 1 deletion hop-hsm-jdbc/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>org.projectdatahopper.hop</groupId>
<groupId>org.hopper</groupId>
<artifactId>hop-hsm-jdbc</artifactId>
<version>0.10.0-SNAPSHOT</version>
<packaging>jar</packaging>
Expand Down
8 changes: 4 additions & 4 deletions hop-marketplace-repo.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@
# plugins: optional metadata that enriches names/categories and can pin version.
# Restart Hop after install so the plugin registry reloads.
#
# Cutover: 0.9.0 and earlier were published as org.apache.hop:hop-datavault.
# Remove plugins/misc/datavault from an old install before installing hopper-edw.
# Cutover: 0.9.0 and earlier were published as org.apache.hop:hop-datavault
# and unpacked to plugins/misc/datavault. Remove that folder before installing hopper-edw.
#
schemaVersion: "1.0"
kind: hop-marketplace-repository
Expand All @@ -51,7 +51,7 @@ description: >
# minHopVersion: only show when the running Hop is >= this version
# version: pin of the published base version (Nexus resolves latest SNAPSHOT timestamp)
plugins:
- groupId: org.projectdatahopper.hop
- groupId: org.hopper
artifactId: hopper-edw
version: 0.10.0
minHopVersion: "2.19.0"
Expand All @@ -61,4 +61,4 @@ plugins:
Apache Hop plugins to build an Enterprise Data Warehouse: source models,
Data Catalog, Data Vault 2.0, Business Vault, dimensional models, quality,
and lineage.
installPath: plugins/misc/datavault
installPath: plugins/misc/hopper-edw
4 changes: 2 additions & 2 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>org.projectdatahopper.hop</groupId>
<groupId>org.hopper</groupId>
<artifactId>hopper-edw</artifactId>
<version>0.10.0-SNAPSHOT</version>
<packaging>jar</packaging>
Expand Down Expand Up @@ -756,7 +756,7 @@
</executions>
</plugin>

<!-- HTML user guide for the plugin zip (plugins/misc/datavault/docs/). -->
<!-- HTML user guide for the plugin zip (plugins/misc/hopper-edw/docs/). -->
<plugin>
<groupId>org.asciidoctor</groupId>
<artifactId>asciidoctor-maven-plugin</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion scripts/docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ COPY --from=build /src/target/hopper-edw-*.zip /tmp/hopper-edw.zip

RUN unzip -q /tmp/hopper-edw.zip -d /opt/hop \
&& rm /tmp/hopper-edw.zip \
&& chown -R hop:hop /opt/hop/plugins/misc/datavault /opt/hop/lib/jdbc /opt/hop/lib/core
&& chown -R hop:hop /opt/hop/plugins/misc/hopper-edw /opt/hop/lib/jdbc /opt/hop/lib/core

COPY scripts/docker/entrypoint-init.sh /opt/hop-datavault/entrypoint-init.sh
COPY scripts/docker/entrypoint-prepare.sh /opt/hop-datavault/entrypoint-prepare.sh
Expand Down
Loading
Loading