08-Ensuring Data Integrity: How to Backup and Archive in SapphireOne

Backup Overview

In the context of SapphireOne’s backup process, it is strongly recommended to store backup files on a drive separate from the one housing your live data file. This precautionary measure is taken to safeguard your data in the event of a drive failure, ensuring that backup data remains accessible from a functioning drive. It is also recommended that you keep additional backups off site.

SapphireOne provides all our clients 333GB of storage space on our SapphireOne File Station storage device which is inclusive with your annual maintenance fees paid on anniversary of purchase. Additional space can be purchased if required.

While the same folder structure that is described to follow can be set up on the alternate drive, it’s essential to understand that a backup is most effective when supported by additional backups. This redundancy further enhances data security and resilience.

SapphireOne’s Files and Folders

In SapphireOne’s Knowledge Base, you will encounter mentions of a designated folder intended for housing your Live Data File. Upon delving into this folder’s contents, you will discover an array of files and folders, each file is identified by its unique file extension, which is defined as the characters appearing after the last dot in the file name. In the list below, please take note that a preceding dot is presumed for each extension:

SapphireOne’s Files

• 4DD – The main data file in the Live Data File folder is the 4DD file. This file is essential and contains the primary data. If necessary, all other files within the Live Data File folder can be regenerated solely from this main data file. However, it’s important to note that for larger data files, the creation of the index file may require some time due to the volume of data involved.
• 4Dindx – One of the files mentioned above is the 4Dindx file. It is worth noting that after the 4DD file, the index file is typically the largest file among the files in the Live Data File folder. The purpose of the index file is to facilitate efficient data retrieval and searching operations within the main data file. Due to its size and the amount of indexing information it contains, the index file may occupy a significant amount of storage space compared to other files in the folder.
• journal – This serves as a 4D backup file that captures incremental changes since the last full 4D backup. It is important to note that if a full 4D backup is not regularly performed, the journal file can grow significantly in size, reaching several gigabytes. In such cases, it is crucial to contact SapphireOne staff immediately for assistance. They can guide you on how to manage the journal file and ensure the proper backup of your data.
• Match – This file is typically very small in size, measured in kilobytes. Its purpose and specific function may depend on the context and configuration of your SapphireOne system. However, due to its small size, it is unlikely to have a significant impact on storage space or performance.

SapphireOne’s Folders

• Log folder – This folder serves as a repository for a text file that diligently records the outcomes of 4D backups, both successes and failures. This log file chronicles the backup activities and offers valuable insights into the backup process, highlighting any encountered errors or issues. Monitoring the Log folders contents aids in tracking the status and reliability of your backup operations.
• temporary files folder – Typically, this folder remains devoid of files during standard operations. This folder is utilised when the SapphireOne/4D application necessitates temporary storage for specific operations or processes. These temporary files are short-lived and serve transient purposes, not intended for long-term retention. Consequently, the Temporary Files folder is commonly empty unless an ongoing SapphireOne/4D operation demands the creation and utilisation of temporary files.

Storing Data Overview

When operating Sapphire Single User or the SapphireOne server, efficient data management is crucial for optimal performance. The central folder to focus on is the folder named Live Data File, We recommend you avoid locating this file directly on your system desktop. It is strongly advised to establish a dedicated folder, preferably named SapphireOne Constants. However, users retain the flexibility to choose their preferred folder name. Once established, this folder should be positioned directly within the SapphireOne directory, thereby efficiently organising all SapphireOne related files.

SapphireOne’s Folder Structure

To maintain a consistent folder structure in SapphireOne, it’s important to follow this recommended structure:

By adhering to this structure, the SapphireOne Constants folder will always remain located below the primary SapphireOne server folders in the directory tree, even during upgrades. When file sorting is configured in alphanumeric order, with files sorted from high to low, the new upgrade will be positioned just above the prior 4D upgrade, which is situated above the SapphireOne Constants folder. This setup ensures a reliable path to the SapphireOne Constants folder, regardless of the number of 4D upgrades performed.

Please note that the screenshot below has been updated to reflect the current build at the time of publication, which is 20.1.

Sub Folders in the SapphireOne Constants folder

Inside the SapphireOne Constants folder, users have the freedom to create sub folders as required. To enhance clarity and facilitate accessibility for SapphireOne support personnel searching for your company’s data file, it’s recommended to establish a sub folder named ALiveDataFile. The addition of the letter A as a prefix ensures that it occupies a prominent position at or near the top of the folder list, even as additional folders are introduced to the SapphireOne Constants folder. The choice to include spaces in the folder name or not is entirely at the user’s discretion, as modern operating platforms accept spaces in both file and folder names.

• ALiveDataFile – The folder housing your data file, “ALiveDataFile,” should exclusively contain the files and folders outlined in SapphireOne’s files and Folders section and nothing else.
  • Desktop Warning – It’s crucial to be aware that unpacking and directly accessing the “name.4DD” file from the desktop can result in the creation of two folders along with all associated files on the desktop. This may lead to a cluttered and disorganised desktop environment. To avoid this issue, it is strongly recommended to unpack the “name.4DD” file into a designated folder and access it from within that folder. This disciplined file organisation approach ensures that unnecessary folders are not created on the desktop, allowing you to maintain a clean and organised desktop environment.
  • Time Taken – When performing a backup of solely the name.4DD file, it’s essential to consider that generating a new index file may consume a significant amount of time to restore, especially for large data files. The index file plays a pivotal role in facilitating efficient data retrieval and search operations within the primary data file.

Data File Backups

• 4D backups- SapphireOne now integrates a backup solution within the server data file function. When configuring the SapphireOne/4D backup function, users will be prompted to specify the backup location. This convenient feature empowers users to define the storage destination for their backups during the setup process. Selecting an appropriate storage location ensures both the security and accessibility of backup files. Documentation for this function starts immediately below
• Manual Backups – In addition to automated backups, it’s highly advisable that users manually create backups of the data file as compressed zip or rar format files when performing upgrades. This extra backup serves as an additional precautionary measure to safeguard the data’s safety and integrity throughout the upgrade process.

Additional Folders

• Additional Folders – Users have the flexibility to create additional sub folders below the SapphireOne Constants folder as needed. These sub folders can be created to further organise and categorise specific files or data related to the SapphireOne application. In the flow chart mentioned earlier, a couple of these sub folders have already been created to demonstrate their usage and placement within the folder structure. Users are encouraged to create additional sub folders based on their specific requirements and organisational preferences to maintain a well-structured and easily navigable file system.

SapphireOne/4D Automatic Backup Procedure

The SapphireOne/4D automatic backup procedure has reached a high level of maturity and has consistently proven its reliability in meeting our data protection needs. While manual backups remain the preferred practice, having a dependable automated backup system serves as a valuable asset.

For more in-depth information regarding 4D backup and user settings, we recommend consulting the official 4D documentation accessible on their website. This documentation offers comprehensive details and guidance on configuring and utilising 4D’s backup system and user settings. If you have specific questions or require further assistance, please do not hesitate to reach out to us.

Log File and Backups

When accessing a new data file in SapphireOne, an alert may appear indicating that it cannot locate the log file. In such cases, the user will be prompted to specify the log file’s location. If the default settings are accepted, the log file will be stored in the same folder as the 4DD files and will be named companyname.journal, as depicted in the screenshot below. BondiBlue.journal

It’s important to note that if backups are not configured, this journal file can grow significantly in size, sometimes reaching up to 20GB or more. This can make data management complex. To disable this functionality, users must uncheck the Use Log checkbox in the User settings for the data file. Detailed documentation on configuring data file settings, including this option, can be found later in this article.

While the backup option is accessible through the File menu for the server, it’s important to note that unless a specific location is specified, the backup will be stored within the application itself. When the backup option is chosen, it will create the backup, and as a result, the journal file size should be reduced to zero.

4D’s Automatic Backup Procedure

4D has taken steps to address the issue of backup details and database settings being lost during server upgrades. The introduction of the Settings folder, located in the same directory as the main 4DD file, is a welcome addition.

This enhanced setup should help maintain critical configuration information and ensure a smoother transition during server upgrades. If you have any specific queries or require further information about using this new Settings folder or any other aspect of SapphireOne, please feel free to ask for assistance or alternately consider utilising our Sapphire AI Assistant.

How and Where is the Settings folder Created

The Settings folder in SapphireOne is created when a backup configuration is established. This can be done by navigating to the Scheduler tab and modifying the Automatic Backup option from Never to another setting, then saving the changes. Upon saving, 4D will promptly generate the Settings folder, as depicted in the screenshot provided.

The Settings folder is located within a sub folder of the Data File folder. It’s crucial to maintain this folder in its designated location within the data file directory. By doing so, you ensure that the server can access the necessary backup information from the Settings folder.

Manual backups

During manual backup processes, users do have the choice to select only the 4DD file. When this option is exercised and the restoration is executed, 4D will generate all the necessary files, excluding the Settings folder. To ensure complete data restoration, it is essential to include both the 4DD file and the Settings folder in the backup.

What is in the settings folder

The Settings folder in SapphireOne serves as a repository for essential information accessible and configurable through the administration window of the SapphireOne server. Inside this folder, you will find the following details:

• IP Address – If an IP address has been entered for the server, it is stored here.
• Port Number – The port number used by the server is also stored in the Settings folder. By default, it is set to 8080, but it can be modified if needed.
• Database Details – Various database-related details are stored, which help in configuring and managing the database effectively.
• Backup Procedure – If a backup procedure has been set up, it is documented within this folder.

Thus this Settings folder plays a pivotal role in preserving and managing vital server configuration information and backup procedures within SapphireOne.

When the server is initiated, the user is prompted to locate the necessary 4DD file. Subsequently, the server searches for the remaining required files. If these files are not present, the server will generate them. At this juncture, the server also checks for the existence of the Settings folder. If the folder is discovered, the server reads the contained details and applies them to the server as necessary.

Upgrading the Server

Each time the server is upgraded the system will try to return to its default settings. The user will have to execute the following procedure to re-initiate the 4D backup procedure.

• No Settings Folder – If there is no Settings the user will have two choices.
  • New Backup – Navigate to the Files dropdown menu and enter the backup settings for the first time, following the documented instructions provided below.
  • Existing Backup – If backups have been previously configured but the Settings folder is missing, such as after restoring a data file, the user will need to re-enter the backup settings.
• Settings Folder – However, when a Settings folder is detected, the server should read the settings stored within the folder. Subsequently, the backup procedure should be restarted. Nonetheless, it is advisable for the user to verify that backups are being executed correctly despite the presence of the Settings folder.

Establishing a 4D Backup Procedure

When setting up a backup procedure for the first time, there has been a change for MacOS-based platforms from version 19 onwards. Users attempting to set up any new backups directly from the Maintenance tab of the Server Administration window, will notice that the backup options From the Maintenance tab on the Server Administration window are greyed out and not accessible.

It should be noted that his change affects MacOS-based platforms only. When initially establishing a new backup procedure for the first time. Users should follow these steps:

1. Navigate to the File menu
2. From the drop-down menu, select: Settings > User Settings for Data File.

Once the User Settings for Data File screen is displayed, users should change the Automatic Backup option from Never to any other desired frequency. At this point, they have two choices:

1. They can proceed to manually enter all of the backup details directly from this screen Immediately and then save it. OR
2. Save the Scheduler screen data with the Automatic Backup option set to anything other than Never.
   • Subsequently, they can then access the User Settings for Data File, screen directly from the Server Administration window.

For Windows-based platforms from version 19 onwards, there is no change in the process of establishing a new backup procedure or accessing an existing backup. Users can follow these steps:

1. Access the Server Administration window.
2. Choose the Maintenance tab.
3. Then, select the Settings button. This action will reveal the User Settings for Data File screen, as shown below.

It should be noted that for both methods of accessing the User’s Settings for Data File screen, it is the same screen, just with different paths to access it. On MacOS, once the backup functionality has been activated for the first time, users for both MacOS and Windows platforms may proceed as desired, either from the Server Administration window or through the File menu on the main toolbar.

Scheduler Tab

The server may initially open on the Interface option. In this case, please proceed by selecting the Backup option as indicated below. Once you’re on the backup option screen, you will find three tabs. To access the Scheduler tab, select it.

Scheduler Data Entry

Depending on the option you choose within the Scheduler tab, the screen will display various relevant options as follows in accordance with your selection.

• Every Hour – No additional options are provided or required for this selection.
• Every Day – To set this option, you only need to specify the time of day when the backup should occur.
• Every Week – When choosing this option, you’ll have two selections to make:
  • Date – Select the day of the week from Sunday to Saturday.
  • Time – Specify the time for the backup to be executed
• Every Month – When opting for monthly backups, you’ll have two choices:
  • Date – Select either the First day of the month, the Last day of the month, or choose a specific day from 1 to 31.
  • Time – Enter the preferred time for the backup.
• Personalised – Selecting this option will display a comprehensive list of choices, including all the above options, allowing you to customise the backup schedule according to your specific preferences.

When you select the OK or Save button from any of the three tabs within Scheduler, Configuration or Backup & Restore, the application will promptly save the entered data and instantly create the Settings folder. This folder will now store the backup information that you have entered thus far.

Configuration tab

The Configuration tab plays a vital role in defining how the server handles backups and their destinations in SapphireOne. It encompasses three distinct areas:

1. Content – In this section, you specify what data or content should be included in the backup process.
2. Backup Destination Folder – Here, you determine the location where the backup will be stored or backed up to.
3. Log Management – This area deals with the management and recording of backup-related logs and activities.

Configuration tab Data Entry

Content area data entry

• Data check box – When performing 4D backups, make sure to select only the Data checkbox.
  • Structure – The Structure file can be downloaded again from SapphireOne whenever necessary.
  • User Structure (only for binary database) – It’s important to note that this checkbox should never be checked.
• Attachments – Prior to selecting the Attachments checkbox, it is advisable to consult with SapphireOne support staff for guidance.

Backup file Destination Folder Area

It is highly recommended that before these backups are activated the user goes and create a specific folder to hold these backups. A common name for this would be 4D backups and should be installed under the SapphireOne constants folder.

When the user clicks the search button on both Windows and Mac platforms, the standard search function will be activated, allowing the user to navigate to any folder on the hard drive. Furthermore, just below this area, you will find information about the currently used space on the hard drive and the available free space.

Log management

In the Log Management section, SapphireOne operates as follows:

1. Backup – At the scheduled time, as set up previously in the procedure, SapphireOne will archive the current 4DD file. This means that any transactions performed during the day will not be included in the backup.
2. Log File – When the log file is activated, SapphireOne creates a journal file that records every single action taken in the data file for the day.
3. Restoring – When the “Restore” function is activated, the user will be prompted to decide whether the log file should be incorporated.
4. Incorporating Log File – If the log file is incorporated, SapphireOne’s restore function has the capability to bring the data file back to the state of the final transaction or action made in the data file for the day, ensuring a complete restoration.

Backup and restore tab

This tab has three areas, General Settings, Archive & Restore and Log Integration which will be documented below.

General area

In the “General” area, you will find several straightforward options to configure:

1. Number of Backups to Keep – Specify how many backup copies should be retained.
2. Backup on No Modifications – (e.g., on a weekend). Decide whether backups should be performed even if no modifications have been made, such as on a weekend.
3. Backup Failure Action: Define the course of action to take if a backup operation fails due to any unforeseen reasons.

Archive area

In this area, it is advisable not to make any changes, as the default settings are typically suitable for most backup needs. The settings in this area pertain to how the archive is created, and this largely depends on personal preference.

The option to split the archive into segments may be considered if the data file archive is exceptionally large, typically in the many gigabytes (GB) in size. As a general rule, when the archive exceeds a couple of GB, it might be a suitable time to consider this approach. Additionally, the size of files that can be sent to the SapphireOne server at Bondi should also be taken into account when making this decision.

Restore and log integration area

In the Restore and Log Integration area, please note the following:

• Restore Last Backup Checkbox – Ensure that this checkbox remains unticked. This prevents the system from automatically attempting a restore, which could potentially fail.
• Integrate the Latest Log Checkbox – When selected, SapphireOne will prompt you to decide whether to include the journal file (log file) in the restoration process.
• Log Integration Error Handling – It is recommended to leave this setting on Strict Mode and avoid making any changes to it.

Reset to structure headings Button

When you select the “Reset to Structure Headings” button in SapphireOne, the software will perform the following actions

1. Delete all data entries made during the backup procedure.
2. Revert to the settings found in a new structure file.

Upon selecting this button, a warning alert will be displayed to the user, informing them of the actions that will be taken.

---

You can review our Blog and YouTube channel for additional information and resources on SapphireOne ERP, CRM and Business Accounting software.