When migrating from a source system to Salesforce using the Apex Data Loader a date becomes the date it was minus 1 day. This happens for all of my objects that contains a date.

The source system is MySQL but the data originally comes from Lotus Notes.

Example:
We are migrating customer information into an Account. In MySQL a query generates x rows where a specific date is one of the field values. The rows are via the Data Loader migrated into an Account using upsert.

Row in MySQL
Company name		Customer start date		Category
Company A		2009-04-23		A

Corresponding values in Salesforce after migration
Company name		Customer start date		Category
Company A		2009-04-22		A

Since this causes problems of different kinds for example when creating reports and just the fact that we get wrong data in the system I would like to have it fixed. Anyone know of a simple explanation?

During our discussion internally we were thinking that it might be a time zone issue between Sweden and America. I have asked Salesforce for an explanation and I will update this post when I get it.

As a test I spent Friday evening at work deleting all the data in our Salesforce environment.

The large amounts mainly consists of Accounts and Accounts Locations. The idea is that you should only have to delete the parent object and all its child objects should be deleted when you do this. Now this works, except for some special cases like Opportunities and Cases that are open.

We had about 4GB of data. 2GB of Accounts Locations and 1,8 GB of Accounts. The rest were Opportunities, Cases, Events and other custom objects.

So to solve the special cases with Opportunities and Case we first deleted the Opportunities and Cases directly.

After that we decided to go for the Accounts. Since Account Location is a child object of the Account object the environment would be empty after this. However it took a very long time.

First, it seems as if it takes longer to delete an Account object that has child objects than it takes to delete them separately. Second, there seems to be a limit on how many records per hour you can get from the computer that runs the deletion job (or any job).

We realized that this would take many hours so we aborted and tried to take the objects one by one instead. So we started with the Account Locations instead. I believe this was a good idea but the rate was slow, between 14000 to 130000 records per hour depending on I don't know what. With the current rate we had we calculated that it would take about 6 hours before the Account Locations would all be deleted. After that we could start with the Accounts.

Now I came up with the brilliant idea to distribute the deletion job. I engaged 6 computers and divided up the csv-files with the Account Location id's into 6 chunks of data. I started a deletion job on 6 computers at the same time with these chunks and the conclusion was what I was hoping for, that the job would process 6 times as fast. The rate was about 150.000 to 180.000 records per hour.

After this we had to delete all of the Accounts as well and I engaged another 5 computers. So we had 11 computers with each a chunk of Account id's to use for the Salesforce Dataloader deletion job. It was about 950.000 records and the entire job was over in about 30 minutes. The rate here was about 250.000 to 300.000 records per hour.

As a conclusion I can highly recommend to distribute the work when working with large amounts of data regarding any kind of job, that is upserting, inserting, deletion or extracting.

To be sure there are no more data left in the environment go check the storage usage on Setup -> Data Management -> Storage Usage. Here you can see how many kB each object takes or a percentage of the total.

I was working with an upsert of a set of records with the Dataloader into Salesforce. The data in the CSV-file looked fine when opening it in Notepad. However, after running an upsert with the data loader the result was that all swedish characters got translated into very strange characters.

The source data resides in a MySQL database. After running a query you can export the result to a csv-file by right-clicking in the result view. Apparently the data was not encoded with UTF-8 which resulted in that when the data was upserted the encoding made it look bad.

The solution to this problem is either to make MySQL encode the CSV-files correctly or by changing a setting in the Salesforce Dataloader so that CSV-files are read and written with UTF-8 encoding. This way the Swedish characters will be preserved in Salesforce.

One thing to note is that the same problem occurs if I try to open the CSV-file in Excel or the Dataloader CSV-viewer.

In a project at work we are migrating a Lotus Notes based system to a cloud computing architechture called Salesforce, which is a so-called 'Platform as a Service' (PaaS) system. So I am spending quite some time now to both get a grip of the cloud computing architecture in general and on the migration project. I will shortly present the cloud computing architecture and Salesforce from my perspective in this blog post and later on the migration experiences I have gotten so far from the work before the actual migration.

Cloud computing:

If you have not heard about this yet its time to start learning. The concept has been around for a while but is rather new to the big mass I would say.

The short definition from Wikipedia is:

"Cloud computing is Internet-based ("cloud") development and use of computer technology ("computing"). The cloud is a metaphor for the Internet (based on how it is depicted in computer network diagrams) and is an abstraction for the complex infrastructure it conceals.[1] It is a style of computing in which IT-related capabilities are provided “as a service”,[2] allowing users to access technology-enabled services from the Internet ("in the cloud")[3] without knowledge of, expertise with, or control over the technology infrastructure that supports them."

According to IEEE Computer Society:

"It is a paradigm in which information is permanently stored in servers on the Internet and cached temporarily on clients that include desktops, entertainment centers, table computers, notebooks, wall computers, handhelds, etc."

Several companies have started to build architectures for these types of clouds like SalesForce, IBM, Microsoft, Google, Amazon, Yahoo, Sun, AT&T, Intel, HP etc. and as far as I know more money are invested as this is considered to be a lucrative business and a very interesting option for many companies.

Salesforce:

If we take Salesforce as the example.
It is a platform-as-a-service system which means that the entire platform is hosted as a service on the Internet with all that comes with it like hardware, software, security, communication, backup, you name it. You pay for the user licenses and any additional help. It is scalable which means that you can go from 100 users of the system to 1,000,000 users seamlessly. They can handle pretty much any type of migration of data into Salesforce. The datamodel is built of objects and properties and has a predefined pretty large set of those. For instance there is a complete CRM-model.

You can administer the environment in a rather advanced way using only clicks. Salesforce also have their own language called Apex - an on-demand development language, which is similar to Java or C#. You can install a plug-in to Eclipse to start using the Salesforce IDE and create applications in this language or queries to the system. The queries are created in a SQL-like way and a part of the web service calls.

Salesforce also have an extensive Web Services API for integration with external systems. Either you can write the calls to these methods yourself or you can use the dataloader application that comes freely from Salesforce to import or export data. The only way to communicate with Salesforce is via web services so the data loader uses these and has its pros and cons. For example error- and session management is handled by default by data loader but has to be implemented in case you decide to create you own web service calls from any other source.

A company having its business critical system on the Internet platform don't have to care about new releases of the system. There are about three releases a year that you will hardly notice.

As you can probably tell by now I am pretty excited about this and at this moment almost only focusing about the good things. The things that I have seen that some people talk about as bad things are the fact that all of the company's information resides on another Internet-based system and how secure this really is?

Another question is if you really save money by migrating a system here. You will still need administrators and developers and users. However you will as I have already said not need to maintain any hardware or software and you will most definately get a much better structure.

I intend to write more about different parts of this system as time goes by. Next I will cover data loader or at least the most important parts of it.