Imports quick steps

Introduction

The import functionality can import the following (user) information:

• Users together with all profile fields, addresses, phones and profile images;
• Advertisements;
• User records;
• References;
• Transfers;
• Payments;
• Tokens.

Importing works fairly easy. Go to system > tools > imports and click on import and select the appropriate data to import, if anything is missing you probably need to give your group the permission to import this data. Subsequently you can upload the csv file and click on save. Then you can click on a new tab called lines where you can decided not to import a line or to change (and correct) it. Finally you can click on the button "Import all results with status ready" to start the import process. Below you will find a detailed description of all fields that can be imported together with a lot of examples.

Be aware: transfers and payments can be imported in two ways. Through system > tools > imports and trough banking > tools > import payments. When importing payments through banking all checks are made that are normally done when a payment is made and the payment is always processed at the current date and time. When importing transfers through the system imports some settings and permissions are not taken into account. No fees are charged on imported transfers, no check is made if the user has sufficient balance, also it isn't checked if the user has the permission to pay the other user using the transfer type. This will make it much easier for admins to migrate to Cyclos, without reconfiguring the whole system after the imports have been made. We recommend to only enable the import of transfers in the test instance and when setting up the system, and disable the functionality as soon as the system goes live.

For a quick start you can copy one of the examples and start importing, if you want to now exactly how things work we recommend you to study the general rules and the rules per header.

Rules CSV or zip file

Rules

The data can be imported using a CSV file:

• The file needs to be in UTF-8 encoding
• The first line of the file has to contain all the headers separated by the configured separator (a comma by default).
• The separator is taken from the configuration of the administrator (Configuration: Localization > List separator).
• Each of the following lines of the csv represents an user, ad, user record, reference, transfer, payment or token.
• All lines need to have the same amount of fields (thus the same amount of separators).
• Any filename can be used for the csv file as long as it has the csv filename extension;
• A field or a line can be left blank.
• It is recommended to place all fields between double quotes, this is recommended because in this case the separator can be used in fields without causing problems. So if you have the comma as the value separator, and you have the text: Hi, how are you doing? You should wrap it like this "Hi, how are you doing?" to prevent the parser from splitting it in two values.
• When using the internal name of a field, make sure to write the internal name exactly the same as in Cyclos, the import is case sensitive.

Images

A zip file can be used to wrap the csv file and (optionally) profile images:

• The zip file should contain only one csv file;
• Any filename can be used for the zip file as long as it has the zip filename extension;
• The zip file may contain multiple image files that can be placed inside any folder within the zip.
• For images the header "images" can be used, and the field should contain the path of the image, e.g.: "images/image1.png, images/image2.png" or "image2.png".
• If you want to upload multiple(profile or advertisement images) images, the images can be separated with a comma, e.g.: "images/image1.png, images/image2.png" or "image2.png". Makes sure to use double quotes!

Custom fields

• The header is the concatenation of the string "field." and the internal name of the field, e.g. "field.remarkdate".
• The values for fields of type list (enumerated) should match the corresponding possible values declared in the field. If a value doesn't match a possible value the import line will be marked with a validation error.
• When importing multiple selection fields it is possible to import multiple values, by separating the values with a comma. Make sure to place double quotes around all values, e.g.: "value1,value2,value6" or "option1, option5".
• When importing a boolean the possible values are "true" and "false", when the field is left blank the default value will be used.

Dates

• Dates and date times are parsed according to ISO 8601 or according to the configuration properties for date and time format.

For example, it is possible to import an ISO date 1994-11-05T08:15:30-05:00 or import 05/11/1994 08:15 according to the following configuration: Date format "day/month/year" and time format "24 hours"

Import process

Step 1 (required): Reading files and validating

The administrator job:

• Select the destination group and mark if necessary to send the activation email(for user imports), select the currency (for advertisements imports) or select the record type (for records imports). The other imports have no specific fields.
• Write the import description.
• Select the file (csv or zip) to upload and click save.

The system job:

• The import will begin reading a zip file or a csv file.
  • If there are problems reading the file, corrupt or has invalid separator, then the import is marked with an 'Invalid' status and a message will be shown describing the error.
  • If a zip file was uploaded then the first state of the import is 'Reading images'. After reading all the images, the import switch to the next state 'Reading csv'.
  • If a csv file was uploaded then the first state of the import is 'Reading csv'. In this state, each line of the csv is parsed and the resulting entity is subject to a preliminary validation.
    • If a line passes the validation its status is set to 'Ready'.
    • If the line has a validation error its status will be 'Validation error' and an the associated error message will be shown.
    • If the number of fields in the line is not the expected then its status is turned to 'Invalid'.
    • If any other strange error occurs while validating the line, the line is marked with 'Invalid' and an internal error message is shown.
• After the system has finished validating all the lines, the import status turns Ready.

Step 2 (optional): Skipping lines

When the import status is 'Ready' (to be imported), we can still make some changes. For example we can select specific lines with the status ready and mark then to be skipped.

Step 3 (optional): Editing lines

When the import is on the 'Ready' status, we can go to a particular line of the import and edit its content, just like if we were changing the csv file but without the need to upload it all over again. For example, if a particular line had a validation error because of the content of one field, we can edit this field of the line and save it with the correct value. The line is submitted to validation and if it succeeds the status will change to 'Ready'.

Step 4 (required): Importing lines

Once the import is in Ready state, it can be imported by clicking at the import button. At this point the import state will change to 'Importing'. The lines will change its status to Imported or Import error or Internal error (in case of a strange error). If the line status changes to Imported then the entity was successfully imported. If the line status changes to Import error, then an error message can be found with the cause of the failure. If the line status changes to Internal error, we should probably check the server logs to identify the problem.

When the import finishes processing all the lines the status will change to Imported or Internal error (in case of a strange error).

Step 5: Archive import

After 30 days, a background task will archive the imports in this way:

• If the import status is not 'Imported', it will be deleted.
• If the import status is 'Imported', then it will eliminate all the import information leaving the minimal following data:
  • Import description
  • Import creation and processing dates.
  • Import statistics (number of imported entities etc..)

User and accounts import

From Cyclos 4.13 onwards, it's possible to import users to several groups. You can define a new column 'group' to indicate the group the user will be imported to.

Headers

Header name	Required	Comment
Basic user fields
name	Required	The user's full name.
username	Required	The user's login name. The username must be unique, an error will be given when the name already exists in the system. In case the login name is an automatically generated (account)number the column is not needed and Cyclos will generate the login names.
email	Required or optional according to the Cyclos setting 'Require e-mail'.	It must be a well formatted e-mail address. If e-mail validation is enabled for importing users (in Configuration 'Validate e-mail on' - Import users) the users will get get the 'pending' status. Be aware that users with pending status cannot have an account, therefore in this case account information is not imported.
creationdate	Optional	If specified the date in the personal member reports (member since) will be set with this date. If no date is set in the column the current (import) date will be set. Check the configuration to see what date format is expected.
broker	Optional	If a broker is specified, it will be assigned as the main broker. Only active brokers are valid.
images	Optional	For profile images the header "images" can be used, and the field should contain the path of the images, for more information see #Images.
group	Optional	Internal name or name of the group the user will be imported into. If a group is not given for a user but a default group was defined in the User import page then the user will be imported to that group.
Address fields In the configuration is set if an address is required. If you want to import more then one address per user an identifier can be used, it should be a unique name e.g.: "address[1].name","address[2].name". If only one address is imported the identifier can be skipped, e.g.: "address.line1". Be aware: the address will not appear right away on the Google map, but a polling tasks executes this task one by one (which might take some time).
address[identifier].name	Optional	The name of the address, when no name is given the address will get the name "address 1", "address 2", etc..
address[identifier].line1	Depends on configuration (required address fields)	The first line of the address (typically street name).
address[identifier].line2	Depends on configuration (required address fields)	Second line of the address.
address[identifier].neighborhood	Depends on configuration (required address fields)	Neighborhood of the address.
address[identifier].pobox	Depends on configuration (required address fields)	PO Box identification.
address[identifier].zip	Depends on configuration (required address fields)	ZIP code of the address
address[identifier].city	Depends on configuration (required address fields)	The city of the address. When it is not specified and the default city is specified in the configuration, the default city will be used.
address[identifier].region	Depends on configuration (required address fields)	The region or state of the address. When it is not specified and the default region or state is specified in the configuration, the default region or state will be used.
address[identifier].country	Depends on configuration (required address fields)	2-letter country code defined in ISO 3166, (must be in capital characters). When it is not specified and the default country is specified in the configuration, the default country will be used.
address[identifier].private	Optional	When the address is set to private other users cannot see this address. When it is not specified the default settings from the configuration are used. Possible options are "yes", "true", "no" and "false".
address[identifier].buildingnumber	Depends on configuration (required address fields)	The building number.
address[identifier].complement	Depends on configuration (required address fields)	The address complement field.
address[identifier].street	Depends on configuration (required address fields)	The name of the street.
Phone number In the configuration is set if an mobile of or landline phone number is required. If you want to import more then one phone number per user an identifier can be used, it should be a unique name e.g.: "phone[1].name","phone[2].name". If only one phone number is imported the identifier can be skipped, e.g.: "mobile.number".
mobile[identifier].name	Optional	Name for the mobile phone number (e.g. private).
mobile[identifier].number	Depends on configuration	The mobile number.
mobile[identifier].private	Optional	When the mobile number is set to private other users cannot see this number. When it is not specified the default settings from the configuration are used. Possible options are "yes", "true", "no" and "false".
landline[identifier].name	Optional	A name for the land line phone number (e.g. "home").
landline[identifier].number	Depends on configuration	The land line number.
landline[identifier].private	Optional	When the landline number is set to private other users cannot see this number. When it is not specified the default settings from the configuration are used. Possible options are "yes", "true", "no" and "false".
Profile fields (custom) Check the products associated to the users group to know which custom fields can be imported. The ones that can be imported are those selected under the column "Enabled" of the "My profile fields" permission.
field.<internalname>	Depends on field settings.	The user profile field, for more information see #Custom_fields.
field.<internalname>.private	Optional	When the custom field is set to private other users cannot see it for this user. When it is not specified ??????????. Possible options are "yes", "true", "no" and "false".
Account fields Some configurations can be specified for each user account.
account.<internalname>.number	Optional	The number for the specified account. In case the user remains pending for activation, the account number is reserved until the user is activated or removed.
account.<internalname>.negativelimit	Optional	The negative limit for the specified account. If empty or not specified the limit will follow what is specified in the products.
account.<internalname>.positivelimit	Optional	The positive limit for the specified account. If empty or not specified the limit will follow what is specified in the products. UNLIMITED or unlimited can be specified to remove the upper limit.
Password fields
password.<internalname>.value or password	Required or optional according to the Cyclos setting in password type (default password type is manual).	The generated passwords (i.e not manual) will not be created if "Send activation email" is not checked, there's a bulk action you can execute to generate and send the passwords to the users. A password is required if it's selected at any channel configuration, it's marked 'At registration' in the products of the importing group and "Send activation email" is not checked. If only one password type is marked 'At registration' you can eventually just use the header "password" for this column, if there are multiple passwords you should use the password internal name too. Be aware: if the password you are trying to import is not used in any channel configuration for the user you are trying to import, then the password is ignored.
password.<internalname>.forcechange or password.forcechange	Optional	Forces the user to change the password upon login. If only one password type is marked 'At registration' you can eventually specify this column as password.forcechange.

Example 1: most simple cases

"name","username","email","password.login.value"
"user1","user1","user1@test.com","1234"
"user2","user2","user2@test.com","1234"
"user3","user3","user3@test.com","1234"
"user4","user4","user4@test.com","1234"
"user5","user5","user5@test.com","1234"
"user6","user6","user6@test.com","1234"

Note:

• In the configuration of this example the list separator is a comma.
• Login password has internal name: login

Example 2: multiple addresses per user

"name","username","email","password.login.value","address[1].name","address[1].line1","address[1].city","address[1].country","address[2].name","address[2].line1","address[2].city","address[2].country"
"Paul McCartney","mccartney","Paul@McCartney.com","1234","Work","69 Royal Worcester Drive","London","GB","Home","Albert Dock","Liverpool","GB"
"John Lennon","john_lennon","John@JohnLennon.com","1234","Work","70 Royal Worcester Drive","London","GB","Home","Albert Dock","Liverpool","GB"

Note:

• In the configuration of this example the list separator is a comma.
• Login password has internal name: login
• Be aware that the addresses will not appear on google map directly, this will be done trough time by a scheduled task, in order to spread the load on google maps and stay within the limits.

Example 3: multiple phone numbers per user

"name","username","email","password.login.value","creationdate","mobile[1].name","mobile[1].number","mobile[2].name","mobile[2].number","landline.name","landline.number"
"Paul McCartney","mccartney","Paul@McCartney.com","1234","25-08-2000","Private","07123456781","Work","07123456782","House","02012345671"
"John Lennon","john_lennon","John@JohnLennon.com","1234","01-01-2001","Private","07123456783","Work","07123456784","House","02012345672"

Note:

• In the configuration of this example the list separator is a comma.
• The date format is "day-month-year".
• The localization country is the United Kingdom.

Example 4: users with profile fields

"name","username","email","password.login.value","field.birthday","field.gender"
"Paul McCartney","mccartney","Paul@McCartney.com","1234","25-08-1982","male"
"John Lennon","john_lennon","John@JohnLennon.com","1234","19-12-1964","male"
"user1","user1","user1@test.com","1234","19-12-1964","male"
"user2","user2","user2@test.com","1234","19-12-1964","female"
"user3","user3","user3@test.com","1234","19-12-1964","male"
"user4","user4","user4@test.com","1234","19-12-1964","female"
"user5","user5","user5@test.com","1234","19-12-1964","male"
"user6","user6","user6@test.com","1234","19-12-1964","female"

Note:

• In the configuration of this example the list separator is a comma.
• Login password has internal name: login
• The date format is day-month-year.
• There is a profile fields with the internal name birthday which is date field.
• There is another profile field with the internal name gender which is an enumerated field with the possible values male and female.

Example 5: users with profile images

Please download the zip file: http://www.cyclos.org/wiki4/files/users.zip

Note:

• In the configuration of this example the list separator is a comma.
• In the configuration the email is set to not be required.

Example 6: users with account configuration

"name";"username";"email";"password.login.value";"account.units.number";"account.units.negativelimit";"account.units.positivelimit"
"user1";"user1";"user1@test.com";"1234";"0000-0001";"1000";"UNLIMITED"
"user2";"user2";"user2@test.com";"1234";"0000-0002";"1000";"100000"

Advertisment import

Headers

Header name	Required	Comment
Basic ad fields
user	Required	The identification value of the user that will own the advertisement.
title	Required	The title of the advertisement.
description	Required	The description of the advertisement.
categories	Required	A comma separated list of categories of the advertisement, must exactly represent the complete path of internal names. e.g. "services>household>cleaning, paymentMethod>cashOnly".
creationdate	Optional	The creation date of the advertisement, when it is not specified the current date will be used as creation date.
publicationbegin	Optional	The day the advertisement will be published and thus visible for other users, when it is not specified the creation date will be used as begin date.
publicationend	Optional	The day the publication of advertisement will end and thus not be visible anymore for other users, when it is not specified, the date will be calculated using the default publication period set in products.
price	Optional	The price of the advertisement, please use the decimal separator as configured in the number format of the configuration.
promotionalprice	Optional	The promotional price of the advertisement.
promotionalperiodbegin	Required if promotional price is set	The date the promotion starts and the promotional price is used.
promotionalperiodend	Required if promotional price is set	The date the promotion ends and the promotional price is not used anymore.
hidden	Optional	A hidden advertisement can't be seen by other users. Options: true / false. When the field is not specified the advertisement will not be hidden by default.
images	Optional	For ad images the header "images" can be used, and the field should contain the path of the images, for more information see #Images.
Advertisement fields (custom) Check the products associated to the users group to know which fields can be imported. The ones that can be imported are those selected in "Enable ad fields".
field.<internalname>	Depends on field settings.	The advertisement field. For more information see #Custom_fields.

Example 1: basic ads import

"user","title","description","categories","price"
"user1","Car","Nice car, owned by an old lady.","goods>car","1399.99"
"user2","Cat","I need to find a new owner for my cat, because he is trying to eat my mouse.","animals>small","100"
"user2","Mouse","I need to find a new owner for my mouse, because my cat doesn't like him","animals>small","55"
"user3","Bike","Brand new bike, never used.","goods>bikes","150"
"user4","Jeans","Used jeans for sale, only has a few holes, but thats popular nowadays.","goods>clothing>other","29.99"
"user4","Shoes","Nice nike air max.","goods>clothing>shoes","59"

Note:

• Make sure the users exist.
• In this example the decimal separator is a dot.
• Ads can only be created in a leaf category. So if "goods>clothing>shoes" exist, you cannot publish anything in "goods>clothing".

User records import

Headers

Header name	Required	Comment
Basic record fields
user	Required	The identification value of the user who will get the user record.
date	Optional	The date the user record is created, if left empty the current date will be used.
createdby	Optional	The identification value of the user who created (filled in) the user record, if left empty the current user that commits the import will be used.
lastmodificationdate	Optional	The last date the user record has been changed (saved), if left empty the user record will not display this field.
lastmodifiedby	Optional	The identification value of the last user that has changed (saved) the user record, if left empty the user record will not display this field.
Record type custom fields
field.<internalname>	Depends on field settings.	The record fields, for more information see #Custom_fields.

Example 1: full import

"user","date","createdby","lastmodificationdate","lastmodifiedby","field.comments"
"user1","16-06-2014 00:00","admin","18-06-2014 00:00","admin","An user record which has a single text custom field."

Note:

• The record type can be selected after uploading the file.
• Make sure the users exist also the createdby user.
• The date format is day-month-year.

Example 2: simple import

"user","field.comments"
"user1","An user record which has a multi line text field."

Note:

• The record type can be selected after uploading the file.
• Make sure the user1 exists.

References import

Headers

Header name	Required	Comment
from	Required	The username of the user that will give the reference.
to	Required	The username of the user that will receive the reference.
level	Required	The level (appreciation) of the user, options: VERY_GOOD GOOD NEUTRAL BAD VERY_BAD
comments	Required	The comment that can be given together with the reference.
date	Optional	The date the reference is given, if left empty the current date will be used.

Example 1: reference import

"from","to","level","comments"
"user1","user6","VERY_GOOD","Good service, thanks!"
"user2","user6","GOOD","This guy is the best :)"
"user3","user6","NEUTRAL","Alright..."
"user4","user6","BAD","He should improve."
"user5","user6","VERY_BAD","Don't trust him!"

Note:

• In the configuration of this example the list separator is a comma.

Payment import

Payments are imported via Banking > Tools > Imports.

Headers

Header name	Required	Comment
Basic payment fields
from	Required	The user who will send the payment, for the user use the username for the system use "system".
to	Required	The user who will receive the payment, for the user use the username for the system use "system".
type	Required	The transfer type that will be used for the payment. Must be in the format: <account internal name>.<tranfer type internal name>
amount	Required	The amount that will be transferred.
description	Optional	The description of the payment.
status.<flowInternalName>	Optional	The status of the payment, the internal neame of the flow must be used.
Transfer type payment fields
field.<internalname>	Depends on field settings	The (custom) payment fields.

Example: simple payment import

"from","to","type","amount","description","status.flow1"
"user1","user2","unitsaccount.tradetransferunits","2","Test 2",""
"user1","user3","unitsaccount.tradetransferunits","3","Test 3",""
"user1","user4","unitsaccount.tradetransferunits","4","Test 4",""
"user1","user5","unitsaccount.tradetransferunits","5","Test 5",""
"user1","user6","unitsaccount.tradetransferunits","6","Test 6",""

Note:

• Make sure there is an account with the internal name "unitsaccount".
• Make sure there is a transfer type with the internal name "tradetransferunits".
• Make sure the date format is "day/month/year".
• Make sure the user1 to user 6 exists.

Transfer import

Transfers are imported via System > Tools > Imports. Be aware: when importing transfers via system imports fees are ignored, the balance of the user is not checked and the user doesn't need the permission to do the payment.

Headers

Header name	Required	Comment
Basic transfer fields
from	Required	The user who will send the payment, for the user use the username for the system use "system".
to	Required	The user who will receive the payment, for the user use the username for the system use "system".
type	Required	The transfer type that will be used for the payment. Must be in the format: <account internal name>.<tranfer type internal name>
date	Optional	When it is not specified the current date will be used as creation date.
amount	Required	The amount that will be transferred.
description	Optional	The description of the payment.
status.<flowInternalName>	Optional	The status of the payment, the internal neame of the flow must be used.
Transfer type payment fields
field.<internalname>	Depends on field settings	The (custom) payment fields.

Example: simple transfer import

"from","to","type","date","amount","description","status.flow1"
"user1","user2","unitsaccount.tradetransferunits","12/06/2014 00:00","2","Test 2",""
"user1","user3","unitsaccount.tradetransferunits","14/06/2014 00:00","3","Test 3",""
"user1","user4","unitsaccount.tradetransferunits","13/06/2014 00:00","4","Test 4",""
"user1","user5","unitsaccount.tradetransferunits","16/06/2014 00:00","5","Test 5",""
"user1","user6","unitsaccount.tradetransferunits","19/06/2014 00:00","6","Test 6",""

Note:

• Make sure there is an account with the internal name "unitsaccount".
• Make sure there is a transfer type with the internal name "tradetransferunits".
• Make sure the date format is "day/month/year".
• Make sure the user1 to user 6 exists.

Token import

Headers

Header name	Required	Comment
Basic token fields
user	Optional	The username of the user to be assigned to the token
type	Required	The internal name of the token type
activatenow	Optional	Boolean value to indicate if the token should be activated or not.
value	Required	The token value
expiry	Optional	The token expiry date.
activationdeadline	Optional	The token activation deadline.

Example 1: minimal token import fields

After import the tokens they will be in Unassigned status and users with properly permission can activate them.

"type","value"
"visa","1234123412341234"
"master","1234123412341235"

Example 2: full token import

"user","type","activatenow","value","expiry","activationdeadline"
"user1","visa","true","1234123412341234","02/07/2060",""
"user1","master","false","1234123412341235","","02/07/2060"