I have set up a WordPress website on a Ubuntu 12.04 Amazon ec2 machine by following this tutorial except the fact that I haven’t created a MySQL database and I am using an amazon RDS MySQL service.

Everything works fine and I am able to create a configuration file during installation but when the installer asks for MySQL credentials. When I fill in the RDS credentials they don’t work.
I even tried entering the credentials manually in the wp-confiq file and still it said it couldn’t connect to the database .

I tried to test out the same credentials on another WordPress installation in a shared hosting environment and the result was the same.

Then on the ec2 WordPress server I installed MySQL and then provided the MySQL details during installation and everything worked fine hence proving its something to do with RDS only.

Additional Details:

1. The DB security Group has only one security: the EC2 Security Group

2. Both EC2 and RDS are located in AWS singapore southeast-1a regions

3. EC2 Security: only port 22 (ssh) and 80 (http) opened

Check that you can connect to the instance with the command line. See instructions here:

http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ConnectToInstance.html

Maybe you have a problem with the credentials or with the network. Once you find the root cause it is easier to solve it.

Check more discussion of this question.

By default you got a username and password for your RDS database. Would it be dangerous to use the credentials on your production website, say in php files?

Or is it necessary to create a user with less control of the db (e.g. only read)? However I can’t find the place to add user on amazon’s console.

Every production site needs to have credentials for the corresponding database they use. These files however shouldn’t be served directly by the web-server.

This means, that for example a .php file shouldn’t be just delivered as plain text BUT it should be interpreted. This you can set in your appache config for the corresponding server and you probably already did if you can see the results of echo phpinfo(); when putting into a php file, and loading it in your browser.

That means, when you give these credentials into the php file, lets say like:

$db_user = <user>
$db_pass = <pass>

It won’t be avaible for the “outside” world. However your application will be able to use these for joining to the appropiate database server.

Check more discussion of this question.

I would like to create a high-available setup (e.g. a small cluster) for a webserver, i.e. it will run Apache, PHP and MySQL.

There will be between 2-8 small websites running with only very little traffic and workload. High availability is however very important.

I don’t want to be dependent on 1 datacenter, so there must be a minimum of 2 servers placed in different datacenters, and if one server goes down, the user must experience no or only a minimum of downtime – and no data loss.

I have considered Amazon AWS using their Elastic Load Balancing, since it is possible to buy 2 EC2 instances in 2 availability zones and set up load balancing and RDS (Multi-AZ).

However this seems rather expensive. Using the AWS price calculator http://calculator.s3.amazonaws.com/calc5.html it totals to 185$/month the first year (including the free tier).

Are my calculations incorrect or is there a cheaper way to make this HA setup?

Best regards

sorry cheap and high-available aren’t two words that have ever gone together. You get what you pay for and generally speaking a service like AWS is going to be the cheapest option you can get if you use a reserved instance.

Check more discussion of this question.

We are using Amazon AWS for our web services but given the recent instabilities in their infrastructure, we are trying to figure out how to run our application across multiple regions for additional redundancy.

Ideally, we would run our entire app in a active-active configuration in multiple regions but our main concern is that we are using RDS, which I understand cannot replicate across regions.

One possible solution (though we have not tried or proven it would work) would be to do mysqldump or EBS snapshots every hour or so but this would mean that we would be forced to run in an active-passive configuration. Our data would be at most an hour behind. This carries its own issues around data synchronization when we failover and the master comes back up, so its not the best solution.

Are there any proven solutions for replicating RDS across regions?

I guess the short answer is: Cannot do this.

If you want to do this, you need to run your own MySQL instances instead of using RDS. Lame answer, and I am surprised that Amazon has decided to not support multiple region replication given their alleged dedication to scalable, redundant and fault tolerant infrastructure.

Oh well :\

Check more discussion of this question.

I have an RDS instance that is costing me a lot of money. From my account activity on amazon I see that the instance has had about 800,000,000 IO requests over the past 7 days.

To give you a little perspective, my app only gets about 6,000 unique visits a day and it doesn’t make that many database connections.

So, what exactly is an IO Request, and why would that number be so unearthly high? I’m willing to do whatever it takes to my app to reduce that cost if necessary, but I’m not sure what’s really going on.

I would appreciate your thoughts.

Ah, Amazon Web Services. One of the biggest moneysinks available. Very good, but not always cheap. Actually, almost never cheap, and always difficult to estimate expenses.

An IO request is a disk-operation. Be that a read, or a write, you’re basically being charged with how much data you’re getting off their system. It’s insane.

You can estimate the IO Requests you’re making, by looking at iostat.

800,000,000 IOrequests in 7 days.. That’s ~=1300 IOPS . Wow. That’s some wickedfast performance.

As an example, a medium sized website database might be 100 GB in size
and expect to average 100 I/Os per second over the course of a month.
This would translate to $10 per month in storage costs (100 GB x
$0.10/month), and approximately $26 per month in request costs (~2.6
million seconds/month x 100 I/O per second * $0.10 per million I/O).

From http://aws.amazon.com/ebs/ about 90% of the way down the page

If you really want to reduce what this is costing you, it might be time to look at whether RDS is really the best fit for your application and project budget.

If you’ve only got 6k hits a day, you might do better on cost and performance with a dedicated server (preferably 2 of em, one for DB, one for web services).
It’s actually a bit more difficult to make that call as you’ve not said too much more about your application.

Check more discussion of this question.

Resetting the RDS master user’s password is simple enough, but how do you find your master users username?

The master user name can be recovered with the rds-describe-db-instances command.

Check more discussion of this question.

We recently moved from a traditional Data Center to cloud computing on AWS. We are developing a product in partnership with another company, and we need to create a database server for the product we’ll release.

I have been using Amazon Web Services for the past 3 years, but this is the first time I received a spec with this very specific hardware configuration.

I know there are trade-offs and that real hardware will always be faster than virtual machines, and knowing that fact forehand, what would you recommend?

1) Amazon EC2?
2) Amazon RDS?
3) Something else?
4) Forget it baby, stick to the real hardware

Here is the hardware requirements

This server will be focused on I/O and MySQL for the statistics, memory size and disk space for the images hosting.

Server 1

I/O
The very main part on this server will be I/O processing, FusionIO cards have proven themselves extremely efficient, this is currently the best you can have in this domain.
o Fusion ioDrive2 MLC 365GB (http://www.fusionio.com/load/-media-/1m66wu/docsLibrary/FIO_ioDrive2_Datasheet.pdf)

CPU
MySQL will use less CPU cores than Apache but it will use them very hard, the E7 family has 30M Cache L3 wichi provide boost performance :
o 1x Intel E7-2870 will be ok.

Storage
SAS will be good enough in terms of performance, especially considering the space required.
o RAID 10 of 4 x SAS 10k or 15k for a total available space of 512 GB.

Memory
o 64 GB minimum is required on this server considering the size of the statistics database.
Warning: the statistics database will grow quickly, if possible consider starting with 128 GB directly, it will help.
This server will be focused on I/O and MySQL for the statistics, memory size and disk space for the images hosting.

Server 2

I/O
The very main part on this server will be I/O processing, FusionIO cards have proven themselves extremely efficient, this is currently the best you can have in this domain.
o Fusion ioDrive2 MLC 365GB (http://www.fusionio.com/load/-media-/1m66wu/docsLibrary/FIO_ioDrive2_Datasheet.pdf)

CPU
MySQL will use less CPU cores than Apache but it will use them very hard, the E7 family has 30M Cache L3 wichi provide boost performance :
o 1x Intel E7-2870 will be ok.

Storage
SAS will be good enough in terms of performance, especially considering the space required.
o RAID 10 of 4 x SAS 10k or 15k for a total available space of 512 GB.

Memory
o 64 GB minimum is required on this server considering the size of the statistics database.
Warning: the statistics database will grow quickly, if possible consider starting with 128 GB directly, it will help.

Thanks in advance.

Best,

The problems:

• The two servers you have listed above are absolutely identical.
• You talk about FusionIO but you also talk about running MySQL and Apache on the same box.
• You don’t mention whether the Apache files or the MySQL database (or parts of it such as the ib_logfile) will be on the FusionIO drives.

The misconception:

It’s not necessarily true that “real hardware will always be faster than virtual machines”. It is true that on the same hardware the same application will perform better for not being in a virtual machine but since you don’t have access to Amazon’s hardware, that comparison is moot.

The point about the cloud is that it scales horizontally, so if you can serve 100 simultaneous visitors with one server, you can serve 1000 simultaneous visitors with 10 servers and each visitor receives the same speed of response, no matter how many of them you have.

The cloud:

There are a few key differences with cloud providers compared to colocation. If you are able to take advantage of them, they will make hosting in the cloud a clear winner.

1. You can spin up and down instances very quickly. If your traffic is very bursty (say, you run a ticket sales website) then you can very easily clone your web tier, database tier and/or storage tier out to hundreds of virtual machines an hour before the Justin Bieber tickets go on sale and shut them all down an hour after to save on money. Hardware based solutions will usually take weeks to increase your capacity and they continue costing money when they aren’t being fully utilised.
2. The up-front cost can be much lower. The hardware you mention probably costs tens of thousands of dollars in addition to your other hosting costs. My Amazon server costs me about $15 per month and yet I could easily scale it up to a much more beefy virtual machine and scale it out to dozens of load-balanced instances with an hours notice.
3. They do a lot of the work for you. Amazon have other services such as DynamoDB which automatically scale out or in to the workload or storage requirements you give it. They run in SSDs for speed and are replicated to multiple places giving you redundancy and availability.

That said, your application has to be capable of scaling horizontally. You can’t simply throw it into the cloud and expect it to scale forever. For instance, default PHP sessions have two problems:

1. They are stored on a local disk meaning you either need to use sticky sessions or a shared disk which will be a bottleneck.
2. They are opened with flock() which is an exclusive, blocking file lock. Only one PHP process can be using a session file at a time. This can be a serious problem when you start firing off lots of AJAX calls.

This is only a single example but applications that have not been written with horizontal scaling in mind are usually full of exclusive resources like that one.

If you are running a distributed database (which Amazon’s database services are) then your app also needs to be able to deal with the trade-offs inherent in the CAP theorem. This states that you can get two of the three aspects: Consistency, Availability, Partition tolerance. You will need to know which of the three you don’t have and have your app compensate for it.

If your application suits hardware, go for hardware. If it suits the cloud, go for the cloud.

Note: I have used Amazon as an example here but there are other cloud hosting providers with similar capabilities of spinning up and down instances very quickly and only charging you for what you actually use.

Check more discussion of this question.

I recently setup mysql 5.5.20 database in Amazon RDS with movable type but actual character encoding doesn’t take effect. I change the parameter group setting to utf8 like below.

character-set-client-handshake = yes
character_set_client  = utf8
character_set_connection = utf8
character_set_database = utf8
character_set_filesystem = binary
character_set_results = utf8
character_set_server = utf8
collation_connection = utf8_general_ci
collation_server = utf8_general_ci
init_connect = SET NAMES utf8

When I build Movable Type 5.13 database table encodeing is cp1252_West_Europe and collation is labeled latin1_swedish_ci but actually should be UTF-8 Unicode and utf8_general_ci

What am I doing wrong?

Okay I got it.

RDS by default sets its encoding to latin8 ignoring the parameter group that you created. So I just needed to delete the first DB that you have to set from the AWS dashboard.

Check more discussion of this question.

I’m managing an enterprise web application that uses Amazon’s RDS as our database server. Our architecture is such that when a user signs up for a new account we create a brand new database for them in RDS.

The database has around 63 tables, most of which have indexes, and almost all have foreign key constraints. Currently we’re creating each new database by executing a large set of sql statements. This takes on average 57 seconds.

Does anyone have any ideas for how I could do this quicker or more efficiently?

You might want to try and execute the query as a “bulk transaction” to skip the verification checks. Be warned, this can create duplicate indexes & break consistency checks if you’re not careful.

Check more discussion of this question.

We’ve been running a large RDS instance in production for over year without a single moment of downtime. Recently, with a bigger budget and higher stakes, we decided to convert it to a multi-AZ instance, to improve reliability and redundancy of our data.

Since then (about two weeks ago), we’ve had two serious failures of our instance which required rebooting. There is nothing in the ‘recent DB events’. There were two DB connection spikes recorded in cloudwatch that occurred at the same time as the failures.

What’s going on?

It appears that the problems were related to AWS and not to the instance. When this was posted, it turned out to be one of the most serious outages of AWS europe so far.

There are a number of problems with RDS Multi-AZ that mean that it is still possible for automatic failover to not occur, leaving you without a server. Implementers should be aware and build sufficient redundancy into their applications that can mitigate problems in multiple availability zones.

Check more discussion of this question.