Contact Us

Call Us

+1 800-78-erwin
(800-783-7946)

Click here for a list of erwin’s global offices.

Technical Support
Submit a Ticket

+1 813-773-4170

Chat

Get live sales support.

Email

Send us comments or
ask general questions.

info@erwin.com

Meta Integration Metadata Management README

 

erwin Metadata Management (EMM) Solutions

erwin Web Portal (EWP)
erwin Data Governance (EDG)

EMM Metadata Management

with Meta Integration® Metadata Management (MIMM)

README for Release Notes, Installation & Setup

Table of Contents

1. Overview

The erwin Metadata Management (EMM) solutions include two products:

  • The erwin Web Portal (EWP) with all essential features for erwin data model publishing, analysis and reviews:
    • Metadata harvesting (integration) from erwin Data Modeler Standard Edition
    • Metadata harvesting (integration) from erwin Data Modeler Workgroup Edition (synchronizes with the erwin Mart repository).
    • Data Model Diagram Visualizer and Navigator
    • Metadata (single) Configuration and Stitching
    • Metadata Browsing, Search and Reporting
    • Metadata Collaboration (external URL, tagging, comments and review)
    • Data Flow Lineage & Impact Analysis
    • Metadata Explorer (simplified metadata user interface for business users)
    • Repository Security with User and Role Management (integration with Enterprise LDAP or SSO)
    • Repository Customization by adding user defined properties / attributes)
    • Repository Storage on PostgreSQL (bundled on Windows), Oracle or SQL Server

    (Note that above features in italic have been added from the previous CA ERwin Web Portal Enterprise Edition)

  • The erwin Data Governance (EDG) product an upgrade from the erwin Web Portal (EWP) product with the following additional features:
    • Universal Data Modeling Web Portal:
      • With integration of third party data modeling tools – including IDERA ER/Studio Data Modeler, IBM InfoSphere Data Modeler, Oracle SQL Developer Data Modeler, and SAP Sybase PowerDesigner.
    • Live Data Store Metadata Harvesting (Data Cataloging):
      • From RDBMS (Relational Databases) technologies – including IBM DB2 & Netezza, Microsoft SQL Server, Oracle, Pivotal Greenplum, PostgreSQL, and Teradata.
      • From HADOOP (Big Data) technologies – including Cloudera, Hortonworks, and MapR.
      • From NO SQL technologies- including MongoDB, Apache Cassandra (DataStax) and Apache CouchDB (MarkLogic).

      • From DATA LAKE technologies – with high level data lake file/directory inventory
        over the cloud (e.g. Amazon S3),
        on big data clusters (e.g. Hadoop distributions),
        or file servers (POSIX/Windows/Linux file systems),
        followed by low level metadata discovery from the database and file import from
        Flat File Database (CSV or Excel)
        JSON (JavaScript Object Notation),
        Apache Avro,
        and Apache Parquet.
    • Change Management:
      • Model Version Management – maintains multiple versions of metadata
      • Repository Configuration Management – associates roles with different configurations of metadata / model versions (e.g. Development vs. Production configurations, or user group focused configurations)
    • Web Metadata Authoring tools:
      • Business Glossary – based on the ISO-11179 standard extended for better data modeling tool integration with Term types like entities, attributes, relationships, domains, business rules.
      • Semantic Mapper – to classify business terms or connect design layers from conceptual to logical and physical data models.
      • Data Documenter – data modeling of existing data stores like RDBMS and big data.
      • Data Modeler – for data modeling requirements with full data modeling from scratch.
      • Data Mapper – for data flow requirements with data mapping specifications.

    (Note that above features in italic are available only in EDG 9.7 or newer)

2. Copyright Notice

The following erwin Metadata Management (EMM) products:

  • erwin Web Portal (EWP)
  • erwin Data Governance (EDG)

are licensed under the following erwin copyright:

All Rights Reserved.

The erwin logo and erwin product names referenced herein are either registered trademarks or trademarks of erwin, Inc. or one of its subsidiaries.
All trademarks, trade names, service marks and logos referenced herein belong to their respective companies.

erwin® is a registered trademark of erwin, Inc.

These erwin Web Portal products are based (re-branded OEM) on the following Meta Integration products:

  • Meta Integration® Metadata Management (MIMM)
  • Meta Integration® Model Bridge (MIMB)
  • Meta Integration® Repository (MIR)

which are licensed under the following Meta Integration copyright:

All Rights Reserved.

Meta Integration® is a registered trademark of Meta Integration Technology, Inc.

Other product and company names (or logos) mentioned herein may be the trademarks of their respective owners.

http://www.metaintegration.com

3. Release Changes

EMM 9.64.2 based on MIMM OEM 9.0.2 (06/10/2016)

  • METADATA MANAGER:
    • Major improvements in MODEL VERSION / CHANGE MANAGEMENT preventing the creation of unnecessary new versions of models if the source metadata (e.g. a database, or data model)
      has not changed since the last automatically scheduled metadata harvesting. This is new feature is achieved by taking advantage of the MIMB’s new capabilities to compare
      the metadata of a newly imported model with a previously imported one in order to detect any change. The major benefit of this new feature is to dramatically
      reduce the disk space in the repository by automatically deleting unnecessary versions.
    • New CONFIGURATION / VERSION CHANGE MANAGEMENT capabilities offering a comparator of versions of configurations.
  • METADATA EXPLORER:
    • Major redesign of the Metadata Explorer UI on both the look & feel and actual capabilities.
    • New METADATA SEARCH/BROWSE with FILTERING AND REPORTING capabilities offering:
      • New Search (or Browse) with Filtering capabilities on any attributes/properties
      • New choice of result display as a classic Google like “LIST”, or a new powerful “GRID” offering a spreadsheet like display of multiple attributes/properties at once.
        Such attributes/properties can be individually selected, reordered, and sorted, basically offering a full metadata reporting solution.
        The results can of course be downloaded as CSV/Excel files.
    • New METADATA EDITING Capabilities at many levels including:
      • Numerous new fast and easy “in place” editing to Rename objects, Edit Descriptions, and more.
      • The new Search/Browse GRID display also offers efficient editing with:
        • TABULAR EDITING of multiple objects at once such as Business Glossary Terms, Data Models Tables, or Table Columns
        • BULK CHANGE of multiple objects at once, where a search can return multiple objects (that can then be selectively subsetted) for which changes can be performed at once (e.g. Change the Security Threat Level to Orange to a set of tables at once)
  • METADATA AUTHORING TOOLS:
    • Common Features:
      • Major improvements in CUSTOM ATTRIBUTES (also known as User Defined Properties) to objects of the Business Glossaries and Data Models
        • Custom Attributes are now common to the MM repository (e.g. Security Threat Level = [ Red, Orange, Yellow, Blue, Green ]
          and are therefore shared between Business Glossaries, Physical Data Models, etc.
          Therefore, having a centralized place for maintenance (e.g. adding a new value: purple)
        • Custom Attributes can now have a default value (e.g. default value is green)
        • Custom Attributes now have a much wider scope to be applied at any level
          from a high level repository object (e.g. harvested Model, Data Mapping, Directory)
          to fine grain model objects (e.g. A Business Glossary / Term, and/or a Physical Data Model / Column)
        • Custom Attributes now have a security group associated to them (e.g. the Security Threat Level custom attribute may only be set by a custom Security Approved group)
      • New AUDIT TRAIL for any changes in objects of Business Glossaries and Data Models, including who changed a given attribute and when
    • BUSINESS GLOSSARY:
      • New Business Glossary editing capabilities are now available to the business users under the Metadata Explorer UI.
        (includes tablet friendly in place editing, as well as HTML formatting of descriptions, etc.)
    • DATA MODELING:
      • Common Features:
        • Major improvements in data modeling DIAGRAM EDITING: entity formatting, relationships editing, automatic layout, etc.
      • PHYSICAL DATA MODEL (data documenter for existing data stores databases, data warehouse, data lake):
        • Major improvements (as part of the above common features)
      • LOGICAL DATA MODEL (brand new feature) (enterprise conceptual/logical information modeler):
        • This new feature is postponed to the next release
    • DATA MAPPING:
      • DATA MAPPING SPECIFICATIONS (Data Flow Mapper)
        • Minor improvements and bug fixes
  • ARCHITECTURE & TECHNOLOGY:
    • Major database performance improvements
    • Updated MIMM Web Services which are now based on RESTful API technology (i.e. therefore removing any security vulnerabilities of the older Axis technology)

EMM 9.64.1 based on MIMM OEM 9.0.1 (12/15/2015)

  • ARCHITECTURE & TECHNOLOGY:
    • 100% Java delivery and installation allowing support for Windows as well as a variation Linux/Unix deployments
      • The Metadata Management Server (EMM) can now be installed on Unix/Linux variations.
      • The Metadata Harvesting Agent (MIMB bridges) can be installed:
        • Locally (co-located with a EMM server on Linux) to run 100% java based bridges including
          JDBC database bridges (Oracle, Teradata, DB2, SQL Server, etc.),
          big data bridges (Hadoop Hive, HCatalog),
          and other popular bridges such as ERwin xml, etc.
        • Remotely on a Windows machine for C++ based bridges and COM API based bridges requiring software SDK running on Windows
          such as ERwin native (.erwin) files.
    • 100% HTML 5 (no more Flash) and tablet friendly look & feel, allowing to run on Mac, Tablets, and more
    • Metadata Explorer Customization to offer a better experience to targeted business users of the customer company:
      • Customize headers, company logos, menus, search categories, home page (with search widgets).
    • Improved Metadata Explorer Search Performance
  • METADATA AUTHORING:
    • Improved DATA STORE DOCUMENTER (PHYSICAL DATA MODEL):
      • Live harvesting and automatic update (with version management) from popular RDBMS or big data hadoop data store
    • Common Data Modeling Capabilities:
      • Data governance integration with Business Glossary including naming standards, reuse of terms, creating of terms on the fly with supervised learning
      • Data modeling graphical diagram editor (relationships, annotations, auto layout, etc.)
      • Full integration (import and export) with data modeling tools like ERwin

EMM 9.6.1 based on MIMM OEM 8.0.3 (05/19/2015)

  • DATA DOCUMENTER (NEW):
    • Full integration (import and export) with data modeling tools like ERwin
  • DATA MODEL DIAGRAM VISUALIZER:
    • New diagram auto layout
    • New dynamic layout of a diagram subset starting from an entity with all related entities with one or two levels of relationships (very useful for large diagrams)
  • SEMANTIC MAPPER:
    • New support for models within a multi-model server source,
      allowing to provide documentation (or glossary) at the high level of a given data model within a multi model server, such as:

      • a Data Modeling (DM) repository server with many data models inside (e.g. ERwin Mart)
  • BUSINESS GLOSSARY:
    • New customizable role driven workflow support (can be turned off) and security enforcement
  • METADATA EXPLORER UI:
    • New integrated presentation of Business Glossary terms related to any data store, including the ability to add/remove BG terms documenting a data store
    • Improved search support for auto complete and objects
  • ADMINISTRATION:
    • New group based security model (as side effect of the new role based Business Glossary workflow)

EMM 9.6.0 based on MIMM OEM 8.0.2 (03/18/2015)

  • ARCHITECTURE & TECHNOLOGY:
    • New support for HTML5 only devices like iPad and other tablets (Flash will no longer be needed)
      for graphically tracing any data flow or semantic lineage, visualizing data model (Diagram Visualizer),
      or editing the enterprise architecture layout (Configuration Manager).
    • Improved search performance MM persistence on SQL Server
    • Java 8 (compiled with backward compatibility with Java 7) compliance (Java 6 is no longer supported)
  • METADATA MANAGER UI:
    • New Business Glossary (for Data Governance Edition only)
    • New Data Documenter (for Data Governance Edition only)
    • New Data Flow Mapper (for Data Governance Edition only)
    • New Metadata Manager look & feel (to match the Metadata Explorer)
  • METADATA EXPLORER UI:
    • Brand New (redesigned from scratch) Business User Interface with simplified search, navigation and presentation paradigms for business users to easily access and understand the metadata assets
    • Replaces the “Metadata Explorer” UI which was a read only version of the “Metadata Manager” UI with simplified metadata for business users.
    • New customizable action menus per repository object type (e.g. open BI report with BI tool by default)
    • New dedicated web pages for tracing data lineage & impact, and semantic definition & usage
    • New access to the Configuration’s Enterprise Architecture Diagram
    • Improved search support for auto complete and objects

EMM 9.5.0 based on MIMM OEM 7.2.0 (11/01/2013)

  • IMPROVED: Support for SQL Server 2012 Databases
  • IMPROVED: Web-Based Architecture Diagramming of Configurations
  • IMPROVED: Configuration Management of Mart Subsets
  • IMPROVED: Semantic Lineage Following Roll-up, Roll-down and Many-to-Many Transformations
  • IMPROVED: Enterprise Authentication (e.g., automatic Windows-based authentication)
  • IMPROVED: “High Fidelity” Diagram Visualizer
  • IMPROVED: Minor enhancements and bug fixes

EMM 9.2.0 based on MIMM OEM 7.1.0 (07/30/2013)

  • IMPROVED: “High Fidelity” Diagram Visualizer
  • IMPROVED: Enterprise Authentication (e.g., LDAP based authentication)
  • IMPROVED: Minor enhancements and bug fixes

EMM 9.1.0 based on MIMM OEM 7.1.0 (04/05/2013)

  • NEW: “High Fidelity” Diagram Visualizer
    • For “Faithful” rendering of the original model diagrams as developed in ERwin
      • Retains original object positions and sizes
      • Properly supports auto size entities
      • Retains display levels for individual objects and the diagram as a whole
      • Retains original entity fonts
      • Retains original attribute data type alignments
      • Retains other shapes and additional documentation on the diagram
      • Separates Logical and Physical Diagrams
  • NEW: Diagram Relationship Analysis allows the user to highlight the PK/FK’s involved in a relationship and even generate the relationship join expression which may be cut and pasted for immediate use in SQL queries.
  • NEW: Powerful Search Language
    • Allowing for sophisticated / advanced search such as:
      Any Words, Exact Phrase, All Words, Exclude Words, Wild Card End, Parent and Child, Exact Name, Object Type, Property Type.
  • NEW: Model Browsing Experience
    • Offering a separate model browse panel simultaneously displayed adjacent to diagrams and lineage
  • IMPROVED: Semantic and Data Flow Lineage & Impact Analysis
    • Direct access to essential analysis:
      • Trace Data Lineage (e.g. views to tables)
      • Trace Data Impact (e.g. tables to views)
      • Trace Semantic Definition (e.g. physical tables to logical entity)
      • Trace Semantic Usage (e.g. logical entity to physical tables)
    • Choice of presentation:
      • Simple List (of ultimate sources and/or destinations)
      • Advanced Graph (of semantic and/or data flow)
    • Choice of direction:
      • Reverse (for Data Lineage or Semantic Definition)
      • Forward (for Data Impact or Semantic Usage)
  • IMPROVED: Data Flow Advanced Analysis
    • Separation of data flow and control flow:
      • Data flow (actual movement of data)
      • Control flow (conditions and filters)
        • Column Control (which directly impacts values)
        • Row Control (which does not directly impact values such as filters)
    • Visualization of Data transformation Type of lineage by using color and/or thickness of lines:
      • gray lines means data flow with no transformation (i.e. pass through)
      • black lines means data flow with transformations (e.g. expression)
      • thick lines means data flow with transformation processes (e.g. ETL workflows that can be analyzed when clicking on such lines)
      • yellow lines means column control flow (e.g. lookups)
      • yellow dashed lines means row control flow (e.g. filters)
      • blue lines means semantic flow
  • IMPROVED: Search
    • Search improvements for SQL Server wildcard/word break searches

EMM 9.0.0.1 based on MIMM OEM 7.0.4 (11/15/2012)

  • MAJOR IMPROVEMENTS:
    • New and Improved LDAP support:
      • Integrated Native and LDAP support (by default)
      • Possible use of LDAP for authentication only, while using (the user friendly UI) Native Role Assignments for LDAP Users
      • Possible use of LDAP predefined groups (with a user friendly “search” on LDAP groups) for automatic LDAP driven Role Assignment
      • User friendly UI to create an LDAP query for a virtual group of LDAP Users (e.g. small data administrator group) with automatic LDAP driven Role Assignment (e.g. to Administrator in that case)
    • Updated search results and filter to display model name along with Model Directory name
  • CRITICAL BUG FIXES:
    • “Failed to upload License File” error message on “Save License” in the License window to initialize Web Portal Database.
    • Import/Export Log display unreadable (occurred only on IE, and only when connected to a Web portal on top of SQL Express)
  • INCREMENTAL PATCHES

    Note: The MITI Support Knowledge base contains information describing updates provided as part of the incremental patch process here.

EMM 9.0.0.0 based on MIMM OEM 7.0.4 (10/12/2012)

  • NEW FEATURES:
    • METADATA TAGGING/ANNOTATIONS/LABELS: Ability for business users to “tag” any metadata with labels, at any granularity level from models down to a particular column
    • USER COMMENT MANAGEMENT: Ability for business users to “comment” on any metadata, at any granularity level from models down to a particular column.
      Model Administrators (data stewards) can then review and manage these user comments (e.g. change status from Candidate to Approved).
      Finally, Model Administrators can push selected comments back into the original tool (e.g. ERwin model)
    • FLOATING LICENSE SUPPORT: offering new concurrent user licensing, with options for one or both of:
      • Named (exclusive) users
      • floating (first come first served) users
  • MAJOR IMPROVEMENTS:
    • METADATA PROFILING: major improvements in accurately representing the original tool metamodel
    • LINEAGE TRACING INTERACTIONS: major usability improvements
    • DATA FLOW LINAGE SUMMARY: greatly improved the summary lineage presentation
    • METADATA EXPLORER SIMPLIFICATION: using context dependent tree navigation, therefore limiting the nesting of left navigation panels
    • TOUCH SCREEN TABLET SUPPORT: with full menus available on Action button as a substitute to the right click menu
    • CROSS WINDOW/TAB OBJECT NAVIGATION: with Show Object in Model Browser, in Diagram Visualizer, in Lineage Analyzer, etc.
    • GENERALIZED WEB LINKS AND BOOKMARKES: to any objects in a Model Browser, in a Diagram Visualizer, in a Lineage Analyzer, as well to the precise context of any actions like search and reporting
  • INCREMENTAL PATCHES

    Note: The MITI Support Knowledge base contains information describing updates provided as part of the incremental patch process here.

EMM 8.2.0.2 based on MIMM OEM 7.0.2 (1/31/2012)

  • Initial Release

4. System requirements

4.1 Important preliminary disclaimer notice on all requirements

The following requirements only define the minimal requirements to run the application server with reasonable performance
based on the provided tutorial, or small business use cases.
The actual requirements for enterprise wide use cases based on larger models and configurations
do require significantly greater resources to obtain acceptable performance.

The following requirements are based on:

  • actual physical hardware (no virtual environment),
  • minimal to no network overhead (assuming both the database and Application servers to be locally installed),
  • vendor’s default install of the current version of their software (with all current service or fix packs),
  • no other applications sharing such hardware (starting from a clean machine),

Any other hardware/software configurations are acceptable as long as they provide the same (or better) results on the provided performance benchmark.
In such case, if any problem is discovered (e.g. scalability or performance issues),
then the customer must be able to reproduce the issue using an environment
that conforms to the minimum performance requirements as defined herein.

Potential known issues include (but are not limited to) the following:

  • actual usable hardware performance on virtual environments (e.g VMWare configuration and licenses)
  • network overhead on remote servers (e.g. bandwidth, proxy, VPN issues, VMWare inter OS network limitations without a proper license, etc.)
  • shared resources with competing applications on the same OS, or between OS on a virtual environment,
  • licensing limitations (e.g. most database server licenses limit the number of usable core/CPU)
  • vendor software known limitations and requirements (e.g. Oracle on VMWare vs Oracle VM)

4.2 Web Client requirements

Users only need an internet browser:

4.3 Application Server Requirements

Hardware Minimum Requirements (based on physical hardware performance, not a virtual environment):

  • 2 GHZ or higher quad core processor
  • 8 GB RAM
  • 10 GB of disk space (all storage is primarily in the database server)

Operating System Requirements:

  • Microsoft supported Windows 64 bit versions (including Windows 2008 Server, Windows 2012 Server, Windows 7, Windows 8.x, and Windows 10).
    • Ensure that installer is executed with full Administrator privilege
    • Ensure that Microsoft .NET Framework 3.5 or higher is installed
    • Ensure that all current Microsoft Windows critical updates have been applied
  • Most popular Linux/Unix 64 bit Operation System Versions (such as Oracle Solaris, Redhat or Mac OS).

Application Server Engine Requirements:

  • Apache Tomcat 7 – 64 bit (bundled)
  • Other Application Servers (such as IBM WebSphere or Oracle WebLogic) require manual install/setup, and are therefore not supported by this version.

Java Runtime Environment (JRE):

  • Oracle JRE 8 – 64 bit (bundled and recommended)
  • Other Java Runtime Environment (JRE) (such as IBM Java) require manual install/setup, and are therefore not supported by this version.

4.4 Database Server Requirements

Hardware Minimum Requirements (based on physical hardware performance, not a virtual environment):

  • 2 GHZ or higher quad core processor
  • 8 GB RAM.
  • 20 GB of disk space (or more as needed for the data)

Database Administrator privileges are required to install/uninstall the database.

The EMM Database Server can reuse your existing Oracle, SQL Server, or PostgreSQL server:

  • Oracle 10g R2 to 12c – 64 bit (recommended for large enterprise, default supported version)
    • The character set of the database must be AL32UTF8 (UTF8); because the Oracle InterMedia Search can only index columns of type VARCHAR or CLOB (not the national variants NVARCHAR and NCLOB respectively)
    • The CTXSYS user must be installed: the installation script can be found in <ORACLE_HOME>/ctx/admin/catctx.sql
    • In order to find out what exact Oracle edition/version is actually installed:

      sqlplus.exe SYS@<DB-NAME> as SYSDBA

      select banner from v$version where BANNER like '%Edition%';

    • In order to find out how much memory is actually available to the Oracle database, it is important to first understand
      how Oracle’s memory is configuration and used:

      • The actual available System Global Area (SGA) memory can be found using:

        sqlplus.exe SYS@<DB-NAME> as SYSDBA

        show sga;

        select * from v$sga;

        select * from v$sgainfo;

      • The actual available Program Global Area (PGA) memory can be found using:

        sqlplus.exe SYS@<DB-NAME> as SYSDBA

        select * from v$pgastat;

    • In order to find out how much processing CPU/Cores is actually available to the Oracle database, query the table
      v$parameter for the value of
      cpu_count, or query the table
      v$license as follows:

      sqlplus.exe SYS@<DB-NAME> as SYSDBA

      select * from v$license;

  • Microsoft SQL Server 2008 R2 to 2014 – 64 bit
    • Database server install needs to support text search
    • The database must be configured to interpret SQL in a case insensitive manner. The case insensitive collation must be Latin1_General_CI_AS.
    • Make sure you apply the current Microsoft patches.

      For example, a Microsoft security update once broke the text search functionality for SQL Server 2012,
      which was then corrected in a newer security update named 2840628(v2).

  • PostgreSQL 9.4 (or newer)- 64 bit
    • libxml is needed, you might need to rebuild PostgreSQL with it.

In general, one must ALWAYS install the latest service packs for a given database version BEFORE creating the EMM database.
E.g., for Oracle 11.2 one is required to apply the patches to upgrade to 11.2.0.3, or whatever is the latest patch level at the time. In addition, Oracle 11.2.0.4 must have patch 17501296 applied.

Virtual Memory:
For a Windows based database server, be sure to either:

  • set the page file size to be managed automatically by OS
  • or it should be at least 3 times the memory or RAM size for the machine.

Thus, you must have more than that much free disk space (at least 3 time the amount of memory or RAM) on the drive where the page file is defined to reside.

5. Metadata Management (EMM) Database Server Setup

The EMM Application Server requires the connection to an existing Database server for metadata storage (metadata repository)

However, a quick install for tests or QA purpose can be achieved by using the bundle PostgreSQL database.

See the section Metadata Management (EMM) Application Server Setup for more details.

The following database setup scripts and instructions assume the following by default:

Database Name: MM

Database User: MM

Database Password: = MM123!

The database name and user name can be changed, and the password should of course be different.

After the product is fully installed and web connectivity has been made, one may connect to a different database by way of the web based user interface at Tools -> Administration -> Database.

5.1 Database on Oracle

Create a user MM and a database MM with the following privileges:

sqlplus.exe SYS@<DB-NAME> as SYSDBA

-- Delete previous user and database if needed

-- DROP USER MM CASCADE;

CREATE USER MM IDENTIFIED BY MM123!;

GRANT CONNECT TO MM;

GRANT CTXAPP TO MM;

GRANT CREATE TABLE TO MM;

GRANT CREATE VIEW TO MM;

GRANT CREATE SEQUENCE TO MM;

GRANT CREATE TRIGGER TO MM;

GRANT CREATE PROCEDURE TO MM;

GRANT CREATE TYPE TO MM;

GRANT EXECUTE ON CTXSYS.CTX_DDL TO MM;

GRANT EXECUTE ON DBMS_LOB TO MM;

GRANT EXECUTE ON SYS.DBMS_LOCK TO MM;

-- If you get the error "Database exception occurred: ORA-01950: no privileges on tablespace 'USERS'"

-- ALTER USER MM QUOTA UNLIMITED ON USERS;

Advanced Oracle 12 DB Administrator may also optimize the KEEP buffer pool. For more details, please refer to:

%EMM_HOME%\tomcat\conf\localhost\MM.xml

5.2 Database on Microsoft SQL Server

5.2.1 Database Requirement 1 – Full-text Search

SQL Server must have the Full-text Search component installed and running.
This can be confirmed by making sure a service called “SQL Full-text Filter Daemon Launcher” (SQL Server 2008) is running
in the Services panel or the SQL Server Configuration Manager.

This Full-text Search component can be added to any existing SQL Server, except for SQL Server Express.
In case of SQL Server Express, the Full-text search component is only available in the “Advanced Services” package:

For 2008 R2, download the “SQLEXPRADV_xxx_xxx.EXE” file.

For 2012, download the “ENU\x64\SQLEXPRADV_x64_ENU.exe” file.

Make sure “Full-Text search indexing” check box is enabled for the MM database.
This can be verified or changed by using the SQL Server Management Studio:
first sign in,
then right click on the MM database and select properties,
finally go in the File area to find the above check box, and restart SQL Server.

5.2.2 Database Requirement 2 – Mixed-Authentication Mode

The Mixed-Authentication Mode is usually set during the SQL Server installation process.

The Mixed-Authentication Mode can be verified or changed by using the SQL Server Management Studio:
first sign in,
then right click on the root of the tree (instance of SQL Server Express),
go to Security, and finally select “SQL Server and Windows Authentication mode”

5.2.3 Database Requirement 3 – TCP/IP Protocol Enabled

The TCP/IP Protocol must be enabled in the SQL Server Configuration Manager for both the named instance and the client protocols
(Make sure you restart the service after changing).

5.2.4 Database Preparation

Login to SQL server as a user with server admin role and execute the following commands to create a database “MM” and a user “MM” with password “MM123!” (or another one):

EXEC sp_configure 'clr enabled', 1

RECONFIGURE

Go

CREATE LOGIN MM WITH PASSWORD = 'MM123!';

CREATE DATABASE MM;

ALTER DATABASE MM SET SINGLE_USER WITH ROLLBACK IMMEDIATE;

ALTER DATABASE MM SET READ_COMMITTED_SNAPSHOT ON;

ALTER DATABASE MM SET MULTI_USER WITH ROLLBACK IMMEDIATE;

ALTER AUTHORIZATION ON DATABASE::MM to MM;

5.2.5 Database Connection

Advanced SQL Server Administrators may define (“hard-code”) a set of TCP/IP ports for SQL Server to run over the network.
However, Microsoft now recommends to run the “SQL Server Browser” service
which can be done either in the Services panel or the SQL Server Configuration Manager.

For more information, read:
How to: Configure Express to accept remote connections

The connection string syntax is:

jdbc:sqlserver://<dbServer>:<dbPortNumber>;databasename=<dbName>

To connect to a named SQL server instance other than the default:

  • If the SQL Server browser service is running:
    • If the named instance is configured to listen on dynamic ports:

      In the installer, specify only the instance name (in the format HOSTNAME\INSTANCENAME) and no port (the port field should be left empty), such as:

      jdbc:sqlserver://localhost\sqlexpress;databaseName=MM;

    • If the named instance is configured to listen on static IP ports:

      The SQL Server instance must be configured to run on a static TCP/IP port and that port must be specified in the installer, such as:

      jdbc:sqlserver://localhost\sqlexpress:1433;databaseName=MM;

  • If the SQL Server browser service is not running:

    In the installer, specify only the instance port, such as:

    jdbc:sqlserver://localhost:1433;databaseName=MM;

Note 1: The default database instance name for SQL Server Express is “sqlexpress, and “sqlserver” for any other SQL Server edition.

Note 2: The default SQL Server TCP/IP port number is 1433.

5.3 Database on PostgreSQL

Login to an existing database as a database superuser or a user who has CREATEROLE and CREATEDB privileges

psql.exe -h <HOST-NAME> -W -U <USER_NAME> -p <PORT> -d <DATABASE_NAME>

-- Delete previous user if needed

-- DROP USER "MM";

-- If the user cannot be dropped due to any ownership issues, you'll need to reassign those objects to another user

-- REASSIGN OWNED BY "MM" TO <OTHER-USER-NAME>;

-- Or drop those objects

-- DROP OWNED BY "MM"

-- Create a user MM with LOGIN privilege

CREATE USER "MM" LOGIN PASSWORD 'MM123!';

-- Create a database MM with UTF8 encoding. You may use a different tablespace

CREATE DATABASE "MM" WITH OWNER "MM" ENCODING 'UTF8' TABLESPACE pg_default;

-- Connect to the database and add an extension

\c "MM";

CREATE EXTENSION intarray;

6. Metadata Management (EMM) Application Server Setup

6.1 Application Server Installation and Configuration

The EMM Application Server is installed as follows:

  • On Windows operating systems, use unzip to extract the software package (.zip) in the directory of your choice.
    You should avoid using the “Program Files” directories of Windows 7, 8.x and 10 as they have are now controlled by Windows with special access rights.
    Depending, on your software installation directory, you may need “Administrator” privileges.
  • On Linux operating systems, use tar -xjvf to extract the software package (.tbz2) in the directory of your choice.
    Depending, on your software installation directory, you may need “root” privileges.

If your are using an existing database and do not wish to customize the application server (e.g. memory allocation, Windows services),
then you can skip this step and go directly to the section on Application Server Execution and Initialization

Otherwise, go to the software home directory and “run As Administrator” the Setup utility (.bat on Windows or .sh on Linux).
This setup utility will allow you to setup the configuration parameters defined below through a user friendly application.
After any change on any panel (tab) below, remember to press the Configure button in order to perform the configuration changes.
A dialog box will be issued to confirm success or failure (with error messages).
Alternatively, this setup utility also works at the Windows command line or Linux shell, use the -help the options.

  • Database Server tab:

    This is to be used only if you wish to use the bundled PostgreSQL database.

    • Enable Windows Service

      This will create the ‘MM Database Server’ Windows Service, set it for automatic start, and actually start it.
      Note that if no database already exists in ‘MM_HOME\postgresql\data\’, then a new database will be created.

      Unchecking that box will delete the ‘MM Database Server’ Windows Service, which is a good idea before uninstalling the MIMM software.
      Note that your existing database in ‘MM_HOME\postgresql\data\’ will not be deleted as side effect.

    • Port Number

      This is set to 5432 which is the default of PostgreSQL, but can be changed to avoid conflicts with other servers.

  • Application Server tab:
    • Enable Windows Service

      This will create the “EMM Application Server” Windows Service, set it for automatic start, and actually start it.
      Unchecking that box will delete the “EMM Application Server” Windows Service, which is a good idea before uninstalling the EMM software.

    • Metadata Harvesting Server Only

      This allows to setup this application server as a metadata harvesting server only, rather than a full metadata management server.
      This is very useful in architecture deployments where the metadata management server is deployed on Linux,
      but needs to access remote metadata harvesting servers (agents) on Windows machine where DM/DI/BI client tools are Windows only (e.g. COM based SDK).

    • Metadata Harvesting Browse Path

      This controls the access to the file system for metadata harvesting.
      The default value is set to ‘*’ which means any Windows drive (C: and any mounted remote drive R:) or any directory from root on Linux.
      It is strongly recommended to limit the access to a common shared data location, and avoid system area.

    • Data Directory

      This is the location of all data files, including log files as well as the metadata harvesting caching.
      The data directory is located by default in the ‘data’ subdirectory of the application server home directory.
      It is recommended to separate the program data from the program files,
      this allows you to provide a new location for the data in a separate area (with regular backups if possible).
      Note that changing to a new location will not move the existing data from the previous location.
      Either the new location already had the data (from a previous install), or new data will be created.

    • Max Memory

      This defines the maximum memory used by Java (JRE) on the EMM Application Server (Apache Tomcat).
      This is unrelated to the maximum memory used by java on bridges for Metadata Harvesting
      which is separately set by default with the M_JAVA_OPTIONS variable in %EMM_HOME%\conf\conf.properties,
      and can be overridden within the Miscellaneous parameter of memory intensive import bridges (e.g. JDBC).

    • Port Number

      This set to a custom port number by default to avoid conflicts with other web application servers.
      However, this can be set back to 80 to avoid having to specify any port number in the URL.

    • SSL

      This enables Secure Socket Layer (SSL) communication for web access (HTTPS).
      In order to support HTTPS, the EMM Tomcat service must be configured to work with HTTPS
      for encryption of passwords and other content exchanged between the web client and the EMM Application Server.
      In this case, you will need a certificate for the HTTPS protocol to work.
      Note: the EMM software does not perform any error handling for validating a certificate associated with the EMM Application Server,
      and most web browsers will report an error if the certificate is not provided by a valid certificate authority.
      Thus, your certificate should be a trusted certificate provided to you by a valid Certificate Authority.

      • Certificate file

        Mandatory

      • Root Certificate file

        Optional (only required if the above certificate file was generated by an external company as a certificate authority)

      • Key file

        Mandatory

      • SSL Key Password

        Optional (only required if the above key file is password protected)

6.2 Application Server Upgrade

6.2.1 Understanding the Data Locations

Most application data is obviously located and your database server, you are responsible for regular backup of such database.

However, it is also important to understand that the software installation directory (known as %EMM_HOME% in this document)
also contains some critical application data and application setup customizations that have to be taken into account in your backup or upgrade process, including:

  • %EMM_HOME%\postgresql\data

    which contains the actual PostgreSQL database data when configured with the Setup utility (in the “Database Server” tab).

  • %EMM_HOME%\data

    which contains other application data such has
    the metadata harvesting cache (critical for incremental harvesting, and metadata export),
    the application server cache (Tomcat),
    the log files (metadata harvesting with MIMB and Tomcat),
    and other temp files.
    Remember that the actual location of this EMM Application Server data directory can be configured with the Setup utility (in the “Application Server” tab).

  • %EMM_HOME%\conf

    with the conf.properties file containing most customizations defined with the Setup utility (in the “Application Server” tab),
    and the \ressources directory containing any User Interface Customizations.

  • %EMM_HOME%\tomcat\conf

    with the tomcat.properties file containing the tomcat port and memory customizations defined the Setup utility in the “Application Server” tab,
    and the keystore file containing the tomcat SSL certificates defined with the Setup utility (in the “Application Server” tab).

  • %EMM_HOME%\jre\lib\security

    which also contains some SSL customizations defined with the Setup utility (in the “Application Server” tab).
    It is recommended to not reuse such directory, but rather reinstall the SSL keys with the Setup utility.

6.2.2 Upgrade Process

We recommend the following upgrade process:

  • Stop the EMM Application Server (possibly using the Windows services).

    If you are using the bundled PostgreSQL database, then you must also stop the EMM Database Server (using the Windows services).

  • Backup your previous installation by renaming the %EMM_HOME% directory as %EMM_HOME%.old

    and then install the new software package at the exact same previous location: %EMM_HOME%

  • Restore your data and customization/setup by copying the appropriate files and directories (as previously explained)
    from %EMM_HOME%.old to %EMM_HOME%,
    including at least \data and \conf\conf.properties
    but possibly more as used and customized such as \postgresql\data, \conf\ressources, or \tomcat\conf.
  • If you are using the bundled PostgreSQL database, then you must first Restart the EMM Database Server (using the Windows services).

    Finally, restart the EMM Application Server (possibly using the Windows services) which may prompt you for a database upgrade of the EMM tables.

6.3 Application Server Execution and Initialization

The easiest way to start the EMM Application Server is to go to the software home directory and use the RestartApplicationServer utility (.bat on Windows or .sh on Linux).

  • On Windows operating systems, you can alternatively use the Windows Services to control the EMM Application Server by using the RestartApplicationService.bat utility instead.
    This utility will create the Windows Service for the EMM Application Server, if it was not already created by previous execution of this utility or the Setup.bat utility.
    At this point, you can simply use the Windows Services to start, stop or restart the EMM Application Server automatically.

    When running the EMM Application Server as a Windows Service, it is important to configure the user running such service in order to have full access rights to the needed files and applications.
    For example, the MIMB bridges involved in the metadata harvesting may need to invoke the SDK of third party software
    such as the COM based API of ERwin, or SAP BusinessObjects Universe Designer.

    In order to set such access rights, go to the services manager of Windows,
    right-click on the EMM Application Server service. Then, go to the “Log On” tab to define an account by name under which the service will run.

  • On Linux operating systems, administrators can use the system daemon directories (e.g. /etc/init.d/ or /etc/systemd/) to control the EMM Application Server
    (either using the RestartApplicationServer.sh utility or directly controlling the tomcat server in the home directory).

The final initialization steps of the setup are performed over the web browser as follows:

  1. Connection

    Connecting to the server on Windows can be simply achieved by opening the Metadata Management link in the home directory.
    In all cases, you can connect to the server using your internet browser to open by default:
    http://localhost:11680/MM.
    Note that the default port of this URL number may have been changed by the Setup utility in the section Server Configuration..

  2. Database

    Define the connection to the previously created database (in the above steps), by providing the database type, user, password, and URL (JDBC connection).
    If you are using the PostgreSQL database bundled with the software package for Windows, then all these parameters should be already preset.
    Press Test Connection button to verify proper database connectivity.
    Finally, when the pressing the Save button, the EMM Application Server will create all the necessary tables in the database.

  3. License

    Click on the Download License Information link to obtain the obtained your HostInfo.xml file that should be sent with your license request.
    Warning: Make sure your are NOT connected to any VPN during that step, then your license will work independently of your VPN connection.
    After you have received your MM.lic license file, browse for it and click on the Save License button.

  4. Login

    Login as “Administrator” with password “Administrator”. Note that you should change that password later in the application by going to: Tools -> Administration -> Users)

6.4 Custom integration with authentication environments

EMM is able to support three authentication methods:

  1. Native Authentication, where the password is managed by the software and stored within the database.
  2. LDAP Authentication, where the software does not manage or store the LDAP passwords at all. Instead, it is simply passed it through to LDAP in order to authenticate.
  3. External Authentication such as Single Sign On (SSO), where the software does not perform any authentication, and leaves that responsibility to a local single sign on service managed by the customer.

In Tools->Administration->Users one may specify either:

  1. Mixed Native and LDAP authentication where users may be authenticated either as native users or LDAP users
  2. External authentication where the system does not perform any authentication, leaving it up to a local Single Sign On environment.
6.4.1 Native Authentication Configuration Issues

There are no specific configuration steps for Native Authentication.

6.4.2 LDAP Authentication Configuration Issues

There are no special server configuration issues for LDAP Authentication. LDAP connectivity configuration is documented in the online help.

6.4.3 Windows Authentication Issues

It is also possible to enable the Application Server to obtains authentication for users from Windows authentication via the browser (client).
This way, users will automatically be authenticated if they are running from a Windows session.

 

To do so, one must install a third party product named Waffle (Windows Authentication Functional Framework) as an addon (see here):

  1. Please ensure that all LDAP settings are correct and users are able to log into the product via LDAP authentication. LDAP connectivity configuration is documented in the online help.
  2. Unzip the Waffle zip.
  3. Copy all the jar files from it to %MIMM_HOME%\tomcat\lib
  4. Open %MIMM_HOME%\tomcat\conf\web.xml. Search for “Windows authentication support”. Uncomment the block following that.
  5. Restart MIMM.
  6. You should have windows authentication enabled now. Any valid windows user will be logged in as guest by default as long as licensing allows it. If you need to get an administrator interface, you can access: http://host:port/Admin
  7. Provide connection information for the database you created above.

 

Note: Waffle is designed around Windows libraries and thus it is recommended that you use a Windows OS based machine as the Application Server. While it is possible to use Waffle on a Linux based machine,
it will require a great deal of manual setup and compilation. Please follow the Waffle documantation for such an implemantion (see here). 

Note: When using Waffle on a Windows based Application Server (as is recommended) you must run as run the MM software as a Windows service (not as an Application) in order for Waffle to work properly. 

Note: Automatic Windows authentication will not allow one to use the browser refresh (f5) with IE 8.x when used as the client browser. Refresh will force a re-authentication on IE 8.x browsers and will not be automatically authenticated.
If this occurs, the user must close all instances of the browser and start again. To avoid this issue, one must use IE 9.x or later or another approved browser
(see System requirements)
In addition, for Internet Explorer and Firefox, you must configure the browser at each client to support automatic Windows authentication.
Please refer to the Waffle web site
here.

6.5 Custom integration for Secure Socket Layer (SSL) communication

Important Disclaimer: SSL is primarily used for HTTPS secure communications from the web browser clients to the EMM Server itself.
Such common HTTPS setup can be fully achieved with the Setup utility as explained in Server Installation and Configuration.
The following steps are provided for illustration purpose only (manual steps), describing what the Setup utility already performs automatically. THEREFORE, YOU DO NOT HAVE TO PERFORM THESE STEPS BELOW.

If you want to manually install a your own certificate, you must:

  1. Change the referenced (in server.xml) connector entry parameters (keystoreFile and keystorePass) to point to the correct keystore file and password.
  2. Import that certificate into the JRE that is being used by this tomcat.
    The default JRE is located under:

    %EMM_HOME%/jre.

  3. Use the following commands:

    cd %EMM_HOME%/jre/lib/security

    move jssecacers jssecacers.old

    %EMM_HOME%/jre/bin/keytool -importkeystore -srckeystore {your_keystore} -keystore jssecacerts

    %EMM_HOME%/RestartApplicationServices.bat

After the configuration, use the default URL to Access EMM: https://localhost:11680/MM

Or use the ports specified in the server.xml file.
For example:

<Connector port="11680" maxThreads="200"

scheme="https" secure="true" SSLEnabled="true"

keystoreFile="conf\keystore" keystorePass="changeit"

clientAuth="false" sslProtocol="TLS" />

6.5.1 Configuring EMM to securely connect via HTTPS to another EMM server for Metadata Harvesting

Important Disclaimer: the following steps are needed ONLY IF you use a self signed certificate for SSL (WHICH IS NOT RECOMENDED),
AND ONLY in the case of configuring EMM to securely connect via HTTPS to another EMM server for Metadata Harvesting.
Only in such exceptional use case, then the following additional steps have to be performed

In order to support HTTPS from a EMM Server acting as the “Metadata Manager” to a EMM Server acting as “Metadata Harvesting” Agent,
the Administrator needs to import the trusted certificate that the EMM “Harvesting Agent” Server is using into the JRE that the EMM “Metadata Manager” server is using.
The following page describes the process:
http://docs.oracle.com/javase/tutorial/security/toolsign/rstep2.html.

The command looks like the following:


cd %EMM_HOME%\jre\lib\security

..\..\bin\keytool.exe -import -alias john -file YourOwnCertificate.cer -keystore jssecacerts

6.5.2 Configuring EMM to securely connect via LDAPS to the Enterprise Directory

In LDAP Authentication, the user password is not managed by the software and is simply passed through to the LDAP system.

Note: this password is not encrypted when communicated between the client and the server.
Thus, in order to ensure encryption you may wish to specify HTTPS protocol communication, as above.

Note: this password is also not encrypted when communicated between the server and LDAP. Thus, in order to ensure encryption you may wish to also specify LDAPS protocol communication
and thus use SSL to encrypt.

In order to support LDAPS, the EMM Tomcat service does not itself need to be configured to work with LDAPS for encryption of passwords. However, to enable secure SSL communication between EMM and LDAP servers
the Administrator needs to import the trusted certificate that the LDAP server is using into the JRE that the EMM Application server is using. The following page describes the process:
http://docs.oracle.com/javase/tutorial/security/toolsign/rstep2.html.

The command looks like the following:


cd %EMM_HOME%\jre\lib\security

..\..\bin\keytool.exe -import -alias john -file YourOwnCertificate.cer -keystore jssecacerts

This is an entirely different certificate from the one used by the HTTPS protocol.

6.5.3 SSL Security Vulnerabilities

Poodle is a “Man In The Middle” (MITM) vulnerability which needs to be primarily fixed server side. An attacker can trick the server into downgrading the encryption protocol used to communicate.
The servers should be configured to disallow TLS fallback, or to disable SSLv3 as a valid protocol.

If Tomcat has been configured with SSL support, the customer should add the following to the connector description in the %EMM_HOME%\tomcat\conf\server.xml

sslEnabledProtocols=”TLSv1.2,TLSv1.1,TLSv1″

7. Model Bridge (MIMB) Metadata Harvesting Setup

The Metadata Integration or Metadata Harvesting from third party databases, data modeling, data integration or business intelligence tools is performed by the integrated Meta Integration® Model Bridge (MIMB) software.
By default, the installer software deploys and configures both EMM and MIMB on the same Windows machine, where the EMM Application Server accesses the MIMB Web Services locally.
MIMB can also be installed and configured as a remote MIMB Agent on another machine: for example on a Windows machine where a Windows only third party software is needed by an MIMB bridge (such as SAP BuinessObjects universe).

Essential customizations (e.g. directories, memory) of the MIMB Application Server can be performed in the following configuration file:

%EMM_HOME%\conf\conf.properties

Recommended customizations include:

  • M_BROWSE_PATH to browse local and mapped network drive.All metadata harvesting file and directory parameter references are relative to the server. The reason is that the server must have access to these resources anytime another event (e.g., scheduled harvest)
    is to occur. When harvesting a model, then, the UI presents a set of paths that may be browsed in order to select these files and directories. Setting the M_BROWSE_PATH parameter allows one to define which drives
    and network paths will be available in the UI. One may update the M_BROWSE_PATH using the UI (on the application server) presented by the setup.bat (or setup.sh on Linux) command
    (see also Application Server Execution and Initialization), or by editing the %MIMM_HOME%\conf\conf.properties file directly.

    On installation, the set includes all directly attached drives., which is specified by an asterisk “*” (M_BROWSE_PATH=*).

    Note for Windows based application servers: When running as a service, the drive names (mapped) and paths may not be the same as what a user sees when logged in, and thus the “*” value will not be see all drives
    you might expect when selecting drives using the UI. Instead, one must explicily list all the drives and network paths that one wants to be available to all users in the UI. Also, it is not sufficient to simply
    enter the mapped drive id (e.g., “N:\”), as that drive mapping is also generally not available to services. Thus, one should specify the physical drives by letters, but must specify the network paths completely, e.g.,:

    M_BROWSE_PATH=C:\, E:\, \\network-drive\shared\

    Note that the above also applies even to script backup and restore drives.

  • M_DATA_DIRECTORY to relocate the data such as the log files, and metadata incremental harvesting cache as needed for very large DI or BI tools
  • M_JAVA_OPTIONS to increase the maximum memory used by java bridges during the metadata harvesting of very large DB, DI or BI tools.
    Note that this parameter defines the default maximum for all java bridges, however most memory intensive java bridges (e.g JDBC bridges) have the ability to define its own maximum memory in their last parameter called Miscellaneous.

8. User Interface Look & Feel Customization

8.1 Login and Headers

Customize the following files and directories using the embedded instructions (in comments):

%EMM_HOME%\conf\ressources\MM.properties

%EMM_HOME%\conf\ressources\web\

8.2 Metadata Explorer for Business Users

Customize the following files using the embedded instructions (in comments):

%EMM_HOME%\conf\ressources\MetadataExplorer.xml