Package org.opends.server.replication.server.changelog.file

Class FileChangelogDB

    • Constructor Detail

      • FileChangelogDB

        public FileChangelogDB​(ReplicationServer replicationServer,
                       String dbDirectoryPath,
                       LogCryptoCfg cryptoCfg)
                throws ConfigException
        Creates a new changelog DB.
        Parameters:
        replicationServer - the local replication server.
        dbDirectoryPath - the path where the changelog files reside.
        cryptoCfg - the cryptoCfg to use for encryption
        Throws:
        ConfigException - if a problem occurs opening the supplied directory

    • Method Detail

      • initializeDB

        public void initializeDB()
                  throws ChangelogException
        Description copied from interface: ChangelogDB
        Initializes the replication database by reading its previous state and building the relevant ReplicaDBs according to the previous state. This method must be called once before using the ChangelogDB.
        Specified by:
        initializeDB in interface ChangelogDB
        Throws:
        ChangelogException - if initialization fails

      • shutdownDB

        public void shutdownDB()
        Description copied from interface: ChangelogDB
        Shutdown the replication database.
        Specified by:
        shutdownDB in interface ChangelogDB

      • clearDB

        public void clearDB()
             throws ChangelogException
        Clears all records from the changelog (does not remove the changelog itself).
        Throws:
        ChangelogException - If an error occurs when clearing the changelog.

      • removeDB

        public void removeDB()
        Description copied from interface: ChangelogDB
        Removes the changelog database directory.
        Specified by:
        removeDB in interface ChangelogDB

      • getDomainOldestCSNs

        public ServerState getDomainOldestCSNs​(Dn baseDN)
        Description copied from interface: ReplicationDomainDB
        Returns the oldest CSNs from the replicaDBs for each replica id in the specified replication domain.
        Specified by:
        getDomainOldestCSNs in interface ReplicationDomainDB
        Parameters:
        baseDN - the replication domain baseDN
        Returns:
        a new ServerState object holding the {replicaId to oldest CSN} mapping. If a replica DB is empty or closed, the oldest CSN will be null for that replica. The caller owns the generated ServerState.

      • getDomainNewestCSNs

        public ServerState getDomainNewestCSNs​(Dn baseDN)
        Description copied from interface: ReplicationDomainDB
        Returns the newest CSNs from the replicaDBs for each replica id in the specified replication domain.
        Specified by:
        getDomainNewestCSNs in interface ReplicationDomainDB
        Parameters:
        baseDN - the replication domain baseDN
        Returns:
        a new ServerState object holding the {replicaId to newest CSN} Map. If a replica DB is empty or closed, the newest CSN will be null for that replica. The caller owns the generated ServerState.

      • getNewestMessage

        public UpdateMsg getNewestMessage​(DomainReplicaId replica)
        Description copied from interface: ReplicationDomainDB
        Returns the newest message from the replica DB for the provided replica.
        Specified by:
        getNewestMessage in interface ReplicationDomainDB
        Parameters:
        replica - the base dn and replica id of the replica
        Returns:
        the newest message from the replica DB

      • removeDomain

        public void removeDomain​(Dn baseDN)
                  throws ChangelogException
        Description copied from interface: ReplicationDomainDB
        Removes all the data relating to the specified replication domain and shutdown all its replica databases. In particular, it will:
        1. remove all the changes from the replica databases
        2. remove all knowledge of the replicaIds in this domain
        3. remove any knowledge of the current generationId for this domain
        Specified by:
        removeDomain in interface ReplicationDomainDB
        Parameters:
        baseDN - the replication domain baseDN
        Throws:
        ChangelogException - If a database problem happened

      • setPurgeDelay

        public void setPurgeDelay​(Duration purgeDelay)
        Description copied from interface: ChangelogDB
        Sets the purge delay for the replication database. Can be called while the database is running.

        Purging happens on a best effort basis, i.e. the purge delay is used by the replication database to know which data can be purged, but there are no guarantees on when the purging will actually happen.

        Specified by:
        setPurgeDelay in interface ChangelogDB
        Parameters:
        purgeDelay - the purge delay. Duration.ZERO disables purging.

      • setComputeChangeNumber

        public void setComputeChangeNumber​(boolean computeChangeNumber)
        Description copied from interface: ChangelogDB
        Sets whether the replication database must compute change numbers for replicated changes. Change numbers are computed using a separate new thread.
        Specified by:
        setComputeChangeNumber in interface ChangelogDB
        Parameters:
        computeChangeNumber - whether to compute change numbers for replicated changes

      • getPublisherFromCookie

        public Flowable<UpdateRecord> getPublisherFromCookie​(String name,
                                                     MultiDomainServerState cookie,
                                                     Set<Dn> excludedDomains)
        Description copied from interface: ReplicationDomainDB
        Returns a publisher of all changes starting from the provided cookie, excluding the changes from the provided set of excluded domains.

        More precisely, it publishes all records with a CSN which is greater than the corresponding CSN in the cookie or are from a domain or replica that is not present in the cookie, except those that are present in the set of excluded domains.

        The subscription to this publisher must be cancelled when it is not used anymore.

        The publisher is automatically updated if some base DN or replica is either added or removed to/from the changelog database, except those that are present in the set of excluded domains.

        Reading will start only when the returned Flowable has been subscribed to.

        Specified by:
        getPublisherFromCookie in interface ReplicationDomainDB
        Parameters:
        name - The name of this publisher
        cookie - Provides the starting point for each replica. If a replica is not present in the cookie, then the starting point is the oldest record of the replica.
        excludedDomains - The domains which are excluded from the publisher. All replicas belonging to these domains are excluded even if they are present in the cookie.
        Returns:
        a publisher of all changes starting from the provided cookie, excluding the changes from the provided set of excluded domains.

      • getPublisherForDomain

        public Flowable<UpdateRecord> getPublisherForDomain​(String name,
                                                    Dn baseDn,
                                                    ServerState startState,
                                                    ServerState endState)
        Description copied from interface: ReplicationDomainDB
        Returns a publisher of changes from the provided domain, using the provided states to determine the starting and ending points of each replica of the domain.

        More precisely, it publishes all records with a CSN which is greater than the corresponding CSN in the startState or are from a replica that is not present in the startState, and with a CSN that is lower or equal than the corresponding CSN in the endState.

        Thus, provided a couple (startCsn, endCns), there are four possibilities for a given replica of the domain:

        • (null, null): publishes ]-infinite, +infinite[
        • (startCsn, null): publishes ]startCsn, +infinite[
        • (startCsn, endCsn) where startCsn <= endCsn: publishes ]startCsn, endCsn]. Automatically removes the replica from the publisher once the endState is reached.
        • (startCsn, endCsn) where startCsn > endCsn: : publishes no record In other words, excludes this replica from the publisher.
        The publisher is automatically updated if some replica is either added or removed to/from the domain, applying theses rules to determine the start and end states for this replica.

        The publisher returns an error immediately after subscription if there is a change with a CSN greater than the start CSN in a replica changelog and no change with the start CSN (which indicates that the corresponding change has been purged). If the most recent change in a replica changelog is older than the start CSN, then it is not an error and the changes will be eventually published just after the start CSN is reached.

        The publisher must be cancelled at the end of use.

        Reading will start only when the returned Flowable has been subscribed to.

        Specified by:
        getPublisherForDomain in interface ReplicationDomainDB
        Parameters:
        name - The name of this publisher
        baseDn - The base DN of the domain to be published.
        startState - Provides the starting point for each replica. If a replica is not present in this state, then the starting point is the oldest record of the replica.
        endState - Provides the ending point for each replica. If a replica is not present in this state, then it has no ending point and its records will be published until the publisher is stopped or cancelled.
        Returns:
        a publisher of changes from the provided domain, using the provided states to determine the starting and ending points of each replica of the domain.

      • getCursorFrom

        public DBCursor<UpdateMsg> getCursorFrom​(Dn baseDN,
                                         ServerState startState,
                                         DBCursor.CursorOptions options)
        Description copied from interface: ReplicationDomainDB
        Generates a DBCursor across all the replicaDBs for the specified replication domain starting before, at or after the provided ServerState for each replicaDB, depending on the provided matching and positioning strategies.

        When the cursor is not used anymore, client code MUST call the DBCursor.close() method to free the resources and locks used by the cursor.

        If the positioning strategy defined in options is GREATER_THAN_KEY or GREATER_THAN_OR_EQUAL_TO_KEY, positioning may happen at a change from a replica added after the matching point, otherwise it will position at a change from one of the known replicas.

        Specified by:
        getCursorFrom in interface ReplicationDomainDB
        Parameters:
        baseDN - the replication domain baseDN
        startState - Starting point for each ReplicaDB cursor. If any CSN for a replicaDB is null, then start from the oldest CSN for this replicaDB
        options - The cursor options
        Returns:
        a non null DBCursor
        See Also:
        ReplicationDomainDB.getCursorFrom(DomainReplicaId, CSN, CursorOptions)

      • getCursorFrom

        public DBCursor<UpdateMsg> getCursorFrom​(DomainReplicaId replica,
                                         CSN startCsn,
                                         DBCursor.CursorOptions options)
        Description copied from interface: ReplicationDomainDB
        Generates a DBCursor for one replicaDB for the specified replication domain and replica id starting before, at or after the provided CSN, depending on the provided matching and positioning strategies.

        When the cursor is not used anymore, client code MUST call the DBCursor.close() method to free the resources and locks used by the cursor.

        Specified by:
        getCursorFrom in interface ReplicationDomainDB
        Parameters:
        replica - the domain replicaId of the replicaDB
        startCsn - Starting point for the ReplicaDB cursor. If the CSN is null, then start from the oldest CSN for this replicaDB
        options - The cursor options
        Returns:
        a non null DBCursor

      • publishUpdateMsg

        public boolean publishUpdateMsg​(Dn baseDN,
                                UpdateMsg updateMsg)
                         throws ChangelogException
        Description copied from interface: ReplicationDomainDB
        Publishes the provided change to the changelog DB for the specified replica id and replication domain. After a change has been successfully published, it becomes available to be returned by the External ChangeLog.
        Specified by:
        publishUpdateMsg in interface ReplicationDomainDB
        Parameters:
        baseDN - the replication domain baseDN
        updateMsg - the update message to publish to the replicaDB
        Returns:
        true if a db had to be created to publish this message
        Throws:
        ChangelogException - If a database problem happened

      • replicaHeartbeat

        public void replicaHeartbeat​(Dn baseDN,
                             CSN heartbeatCsn)
        Description copied from interface: ReplicationDomainDB
        Let the DB know this replica is alive.

        This method allows the medium consistency point to move forward in case this replica did not publish new changes.

        Specified by:
        replicaHeartbeat in interface ReplicationDomainDB
        Parameters:
        baseDN - the replication domain baseDN
        heartbeatCsn - The CSN heartbeat sent by this replica (contains the replica id and timestamp of the heartbeat)

      • notifyEclExcludedDomainsChanged

        public void notifyEclExcludedDomainsChanged()
        Description copied from interface: ChangelogDB
        Notifies the change number indexer that the list of excluded domains has changed.
        Specified by:
        notifyEclExcludedDomainsChanged in interface ChangelogDB

      • getRemovedReplicasCsns

        public RemovedReplicasStates getRemovedReplicasCsns​(Dn baseDn)
        Description copied from interface: ReplicationDomainDB
        Returns the latest known CSNs for removed replicas.

        A removed replica needs to remember both the CSN of the last sent message, which can be a ReplicaOffline or an LDAPUpdateMsg depending on the version it was running, and the last LDAPUpdateMsg message to support old servers talking to it.

        Specified by:
        getRemovedReplicasCsns in interface ReplicationDomainDB
        Parameters:
        baseDn - the baseDn of the replicated domain
        Returns:
        the latest known CSN for removed replicas

      • addMonitorObjectClassNames

        protected void addMonitorObjectClassNames​(Attribute objectClassAttribute)
        Description copied from class: MonitorProvider
        Adds to the provided attribute the objectclass names that should be included in the monitor entry created from this monitor provider. This method should be implemented by child classes to make it easier to search for monitor entries of a specific type.
        Specified by:
        addMonitorObjectClassNames in class MonitorProvider
        Parameters:
        objectClassAttribute - the objectClass attribute where to add the object class names

      • bindTo

        public void bindTo​(MeterRegistry parent)
        Description copied from class: MonitorProvider
        .

        Implementation considerations

        Implementations MUST set the MonitorProvider.registry field by creating a new MeterRegistryHolder, and specifying the appropriate prefix for the dimensional model and also the hierarchical model if they are different. This can be done using a parameter on the constructor, or the MeterRegistryHolder.hierarchicalNamePrefix(String) method. Tags common to all metrics for this monitor provider can also be set using the MeterRegistryHolder.tag(String, String) method.

        Attribute names are then registered by calling factory methods on the MonitorProvider.registry field.

        Sample dummy implementation

        (Please remove the comments when using this code as a template) 

         
 public void bindTo(final MeterRegistry parent) {
     // create the MeterRegistryHolder wrapping OpenDJ's MeterRegistry
     registry = new MeterRegistryHolder("dimensional-name-prefix", parent)
              .tag("label", "dimension")
              .hierarchicalNamePrefix("hierarchical-name-prefix.dimension");

     // register the metrics against the wrapper
     registry.numberGauge("gauge", numberSupplier); // use lambdas or method references here
     timer = registry.timer("requests-submitted");
     // set a different name for the metric in cn=monitor, dimensional model and hierarchical model
     Gauge.doubleGauge(dimensionalName, supplier)
          .hierarchicalName(hierarchicalName)
          .tag(MeterRegistryHolder.TAG_ATTRIBUTE_NAME, "cn-monitor-name") // drop the "ds-mon-" prefix here
          .register(registry);
     // this gauge will only output to cn=monitor based monitoring endpoints
     registry.monitoringGauge("only-visible-in-cn-monitor", supplier);
 }
        Specified by:
        bindTo in interface MeterBinder
        Specified by:
        bindTo in class MonitorProvider

      • renameHeadLogFileToUseOldestKey

        public static void renameHeadLogFileToUseOldestKey​(Path headLogFilePath,
                                                   LogCryptoCfg cryptoCfg)
                                            throws IOException,
                                                   ChangelogException
        Renames the head.log files to a name using the oldest key in the file. If the file is empty, it will be removed.

        Note: This is used by upgrade only. It was decided to put this method here to avoid changing the visibility of a lot of APIs to public.

        Parameters:
        headLogFilePath - path to the head log file
        cryptoCfg - the crypto configuration to be used to read the changelog files
        Throws:
        ChangelogException - if any problem occurs with the changelog
        IOException - if any I/O problem occurs

      • migrateOfflineStateToChangelog

        public static void migrateOfflineStateToChangelog​(Path replicationDbDir,
                                                  Path replicaIdDir,
                                                  LogCryptoCfg cryptoCfg,
                                                  CSN offlineCsn)
                                           throws ChangelogException
        Used by upgrade only: Append the CSN at which the server/domains was considered offline if this CSN appears to be more recent than the last message already contained in the log, otherwise the changelog is not modified.
        Parameters:
        replicationDbDir - Root path of replication files.
        replicaIdDir - Path to the folder were the change log for a specific server/domain is stored.
        cryptoCfg - the crypto configuration to use for optionally encrypting/decrypting the content.
        offlineCsn - CSN at which the server has been considered offline.
        Throws:
        ChangelogException - If any problem occurs.

      • migrateEncryptedHeadLogFile

        public static void migrateEncryptedHeadLogFile​(Path headLogFilePath,
                                               LogCryptoCfg cryptoCfg)
                                        throws ChangelogException,
                                               IOException
        Used by upgrade only: Rewrite the head log file to take confidentiality settings into account.

        Previous versions relied on secret keys being global to the server. To keep confidentiality working, the log file should now contain the secret key in the header, so we create a new one and copy all records re-encrypting them with the new secret key.

        Parameters:
        headLogFilePath - path to the head log file
        cryptoCfg - the crypto configuration to be used to read the changelog files
        Throws:
        ChangelogException - if a problem occurs while modifying the changelog file
        IOException - if a problem occurs when renaming the changelog file

      • hasReplicaDb

        public boolean hasReplicaDb​(DomainReplicaId replica)
        Description copied from interface: ReplicationDomainDB
        Returns true if a changelog already exists for the replica in the domain.
        Specified by:
        hasReplicaDb in interface ReplicationDomainDB
        Parameters:
        replica - the DomainReplicaId to describe
        Returns:
        true if a changelog already exists for the replica in the domain