Caused by POSTing a form to a statically cached page.
nginx will thus return a 405 method Not Allowed.
A nice little trick. Whilst in the console, just type..
>>reload!
& all your application will be reloaded.
You will have to reset any model values as these will still have the old values.
Here is a simple routine that returns a nice friendly elapsed time message; just like the ones you see in all those fancy social networking sites & twitter.
def elapsed_time(start_time)
diff = Time.now - start_time
case diff
when 0..9
'just a moment ago'
when 10..40
'less than ' + (diff * 1.5).to_i.to_s.slice(0,1) + '0 seconds ago'
when 41..60
'less than a minute ago'
when 60..120
"1 minute ago"
when 120..3540
"#{(diff / 60).to_i} minutes ago"
when 3540..5400 then
'about 1 hour ago'
when 54010..86400 then
"about #{(diff / 60 / 60 * 1.02).to_i} hours ago"
else
start_time.strftime("%H:%M %p %B %d, %Y")
end
end
At tickex we have an simple API & quite a few clients using it. I've been a bit suprised by the number of programmers who have asked for sample code to access an XML API using HTTP get. So here is a simple example.
url = URI.parse('http://www.tickex.com/api/events')
user = your_user_id
pwd = your password
request = Net::HTTP::Get.new(url.path)
request.basic_auth(user, pwd)
response = Net::HTTP.new(url.host, url.port).start do |http|
http.request(request)
end
puts response.body
Thinking its time to delve into Cucumber, I have to say found it a bit difficult to grasp what was going on – hence this post.
What is Cucumber? - "a tool that can execute feature documentation written in plain text". So for BDD [Behavioural Driven Development – where feature descriptions & tests are typically written before the code], Cucumber is intended to help non-programmer types such as business analysts, business specialists and the like write tests.
This post covers how to install and write your first, and very basic, Cucumber test.
Webrat is also installed for feature testing websites - be they rails, php, .net, java, or anything else.
Install Cucumber into your rails site
[Windows users can ignore the “sudo” bit]
>sudo gem install cucumber
>sudo gem install term-ansicolor treetop diff-lcs hpricot
Now, Go to your project directory & execute the following:
>ruby script/plugin install git://github.com/aslakhellesoy/cucumber.git
>ruby script/plugin install git://github.com/brynary/webrat.git
You will need to install rspec if not already [read rspec on rails documentation to complete install]
>ruby script/plugin install git://github.com/dchelimsky/rspec.git
>ruby script/plugin install git://github.com/dchelimsky/rspec-rails.git
Bootstrap cucumber
>ruby script/generate cucumber
This will create:
features/steps
features/steps/env.rb
features/steps/common_webrat.rb
lib/tasks/cucumber.rake
script/cucumber
You will write features descriptions in features/my_feature.feature
And the steps to test these fixtures in features/steps/my_feature_steps.rb
Have a quick look at features/steps/env.rb – you can see the options available & how the environment is setup
Have a quick look at features/steps/common_webrat.rb – you can see all the appropriate “When I...” statements.
Now lets run the rake task to see test the install & confirm that something happens.
>rake features
On doing so, I received this error:
DEPRECATION WARNING: Inflector is deprecated! Use ActiveSupport::Inflector instead. See http://www.rubyonrails.org/deprecation for details. (called from /home/matt/development/tickex/vendor/plugins/active_merchant/lib/active_merchant/billing/integrations.rb:11)
I've got an old version of Active Merchant installed.. so time to update
>ruby script/plugin install git://github.com/Shopify/active_merchant.git --force
If all has run successfully you should see coloured Cucumber messages – successful steps are now in green, pending steps in yellow & failures in red. The backtrace can be found next to the failed step, instead of at the end as it used to to be in rspec.
Now lets create our own basic test. We are going to test that a user can browse to the home page. So create a new file called features/home_page_links.feature & fill it like so..
Feature: Home page links
In order to browse my_site.com
A user
Should be able to browse to the home page
Scenario: Browse to the home page
Given I am on the home page
Save the file then from the command line in the applications root directory run
> rake features
You should see something similar to:
Feature: Home page links # features/home_page_links.feature
In order to browse tickex.com
A user
Should be able to browse to the home page
Scenario: Browse to the home page # features/home_page_links.feature:6
Given I am on the home page # features/home_page_links.feature:7
1 steps pending
You can use these snippets to implement pending steps:
Given /^I am on the home page$/ do
end
The “1 steps pending” indicates that you have a feature description, but no test steps. So lets add that.
Create a new file features/steps/home_page_links_steps.rb and place this content in the file.
Given /I am on the home page/ do
visits "/"
end
The Given.. block, corresponds to Scenario statement in the feature file. We are using Webrat to perform the browsing. Now run
>rake features
You should now see something like..
Feature: Home page links # features/home_page_links.feature
In order to browse tickex.com
A user
Should be able to browse to the home page
Scenario: Browse to the home page # features/home_page_links.feature:6
Given I am on the home page # features/steps/home_page_links_steps.rb:1
1 steps passed
with “1 steps passed” being green to indicate all is good.
This should be enough for you to get Cucumber up & running with your rails site and to start making sense of the Cucumber examples.
Here is how to run a programme in the background on linux.
>command > output.txt 2>&1 &
I'm moving this site as its sitting on a really old server on a cox communications DSL link in Laguna Beach - with some of the dodgiest wiring I've ever seen - including India & Japan. I'm not sure how long that's going to be around for & I wanted to have a permanent place where I can experiment.
Dreamhost were offering unlimited bandwidth & disk space for $5/month.
Looking forward to lots ot tedious config error troubleshooting & all that.
Here is a list of the errors so far and how I've resolved them.
1. error id: "bad_httpd_conf"
Unresolved as of the moment..
Here is a great article describing how Jim Meyer, Manager of Linked In's Light Engineering Development scaled his facebook application, Bumpersticker, to over 1 Billion page views per month.
In a nutshell, his three design guidlines are :
- Talking to spindles is bad - talking to file systems and databases has to be minimized
- Dynamic content is your enemy. Anything you can turn into static, including periodic re-caching should be removed.
- Push everything you possibly can to the client to reduce the amount of traffic going over the network no matter where the client is.
I like to build code quality in the culture of development team. Being proud of the code quality is something that I want my developers to aspire to. I actively manage towards this by focusing on shipping high quality production code; not on technical issues [I'm pretty agnostic when it comes to technology].
The three management tools I use to achieve this are:
- Agile development methodologies
- Building code quality into programmer KPIs
- Make programmer KPI measurements public
Agile Development Methodologies
By this I mean:
- Small milestones of production quality code & all stakeholders having visibility of project progress
- Programmer commits own tests with code
- At least a daily build, or preferably a build with every commit & the team is notified of a commit that breaks the build; ie: cruisecontrol
- Daily team meetings [ie:scrum] that update on progress
Stake holders should be able to see an up to date view of project progress & quality. Issues should be identified early & remedial action can be taken before a small issue becomes a large issue.
I like to have a stated code testing/quality assurance strategy. With Tickex, we had commercial pressures to ship code very fast, & change often so I settled up on the follow [its tailored for a Ruby on Rails project]:
- Standardise environments [development, test, stage, production]. Developers can use their preferred tools, but commited code must work on the standardised environment.
- Aim for 100% Rspec Model coverage
- Cruisecontrol runs tests on every commit & notifies team if build is broken
- Utilise Selenium for user land testing, rather than testing controller actions.
- Performance testing using Hpricot & Selenium Grid - - to script concurrent user sessions
- Manual tests for muliple browsers [ok as long as the feature set is not massive]
This is a balances providing the high level of testing coverage with the ability to rapidly turn around features. As a result of this, we have experienced zero downtime from code issues, and very few bugs have made it into production [& most of those have been specification issues, rather than code].
Every environment is different, but here are some guidelines. Each metric on its own is pretty much worthless, but the whole will build a picture.
Quantative KPIs
- lines of code
- milestones met or features delivered
- bugs in code
- number of times commits break build
Qualitative KPIs
- Peer reviews - both quality of code & as a team member
- Customer/Stakeholder feedback - satisfaction with work
- Test coverage of code
- Team fit
Make programmer KPI measurements public
Making these measure available public, on a wiki, & in regular team meetings is intended to build a team spirit. Peer pressure can be a better motivator than agressive management.
I don't favour an agressive leadership style. Fear is one motivitor,but I don't believe it leads to habits being formed & long term behaviourial change. I think that respect of a manager & peer pressure lead to long lasting habits being formed. This is not to say be a push over, think hard but fair; If someone is not performing, I put the ownus upon the team to sort the issue quickly, then act accordingly and decisively if needed. The message will be seen that code quality cannot be compromised.
This philosophy is independent of team size or technology. Whether a small ruby/php team or a large .Net or Java team it does not matter to me. What matters is predictably shipping high quality production code that meets customer requirements.
Google's new Ajax libaries API is a great way to help offload bandwdith & speed up your sites performance. Google hosts the most popular open source Javascript libraries on their content distribution network architecture. By using the Google AJAX API Loader's google.load() method you can access:
Here is a great article describing how to use this in Rails, although it will soon be in plugin form I believe.Matt Cutts has a thread on his blog asking for what Google's web spam team should work on next. It makes for very interesting reading.
Topics of note:
- Adwords & Blogspot spam will probably soon be targeted
- Link farms and paid links are still working for people & Google can't seem to get onto of this.
Hopefully Google & Matt Cutts will start communicating back to the community a little better after this process.
Google can now index flash
Well, kind of, sort of.. but its definitely better than it was.
Binary content such as images & FLV's will probably always remain invisible to the google spiders.
I think the real bummer is that if I build anything with the two most interesting client side technologies, flash & javascript - it won't get spidered & I won't get search engine traffic.
At Tickex we have an API that is accessed via HTTP Basic Authentication.
Here are a few links for developers to read a little bit more about this.
c# dotnet
java
ruby - it's just one line using open_uri - open("http://www.example.com", :auth_info => [user, pwd]) do |f|
PHP
Ruby is awesome for this kind of stuff - a bit high in CPU usage, but your done in a few minutes.
It's been awhile since I used IE & had forgotten that it does not handle MIME Type XML very well. IE will treat an XML file as file to be downloaded; which if fine as you can save the file & look at in in notepad or a programming editor. It did cause me some confusion as I told it a client that it would render fine in their browser :-(
Try this url in Firefox, Safari, or Opera & they will actually render the data - Firefox is best as it keeps the XML formatting.
Google's documentation for the appengine is a bit basic for those not to familiar with Python, like me.
If your building a mashup, your going to want to use the fetch routine to pull down data from a web service. This is one of those vaguely documented features.
Here is how it's done.
import base64
from google.appengine.api import urlfetch
url = "www.target_webservice.com"
username = "your_username"
password = "your_password"
base64string = base64.encodestring('%s:%s' % (username, password))
headers = {'Authorization': "Basic %s" % base64string}
result = urlfetch.fetch(url, headers=headers)
if result.status_code == 200:
all is good
else:
something went wrong
If you compare the open source Content Management Systems available in rails, with those availble in PHP, Java, Python [ie: Django], and even dotnet, I think that you'll be disappointed by the feature sets of the rails
software.
Although lovdbyless & the community engine plugin, whilst not strictly being CMS's provide a social network/blog straight out of the box with a feature set that exceeds the PHP CMS's - for the social networking side of things.
This is not such a big surprise. Firstly the majority of the open source CMS's [ie: Joomla & Django] are quite mature compared to the rails options. Secondly, and much more importantly [having battled with Joomla quite a bit] .. I think the Rails way would be to build it yourself.
Restful authentication, Restful scaffolding, layouts, plugins, & the mentioned above make it possible to build something quite quickly.
Building it yourself means your not constrained by what someone else thinks you needs. The flipside is you need to have the chops to develop what you want. Just as importantly, you need to know what you want.
On the ruby on rails home page there is a screencast of someone building a blog in 15 minutes. Its not very pretty & its not something I would ever put into production use, but you get the idea of rapid development.
Gentle reminder:
Never, ever point your test environment at your production db.
Here are the comments from database.yml
# Warning: The database defined as 'test' will be erased and
# re-generated from your development database when you run 'rake'.
# Do not set this db to the same as development or production.
enough said.
Google, MSN, & Yahoo spiders, or crawlers, follow links on web pages.
If there are no links to a page, it should not be crawled by a search engine. Spiders generally don't look at a url & guess what another link should be. That is usually indicative of something a little more nefarious.
You may have links that used to exist, but do not now. These could be from removed the content; or from dynamically generated your urls & the information used to generate those pages no longer exists.In this case, the links will still be in the search engines index & polled every so often. This is not a bad thing, as the link will eventually fall out of the search engines index. You can request removal through the webmaster admin tools if this bugs you.
Yahoo's spider deliberately generates a dummy url to generate a "404 - page not found" error to understand how your site handles this. Don't be too worried that Yahoo is spidering weird links for you site.
If you feel you need to, you can trap these requests in your "page not found" web site code.
user_agent = request.user_agent.downcase
if [ 'msnbot', 'yahoo! slurp','googlebot' ].detect { |b| user_agent.include? b }
# This is a request from a search engine ..do something here..
end
When trying to have a value auto selected, I spent ages trying to understand why this was not working:
Select_tag(:field, options_for_select(@hash), selected _item)
After a little more CAREFUL reading of the documentation & the source code, this did the trick.
Select_tag(:field, options_for_select(@hash, selected_item))
I'm pretty happy with that.
This means that Google has spidered and indexed almost 100% of Tickex.com.
As all the pages are dynamically generated, it means that my site architecture is very google friendly.
All the pages have unique and meaningful title, meta, h1, h2, alt & a title tags.
The link building programme is another matter, but we are nicely positioned for the long tail.
The ruby daemon gem is absolutely brilliant.
It makes creating a ruby daemon simply as pie.
On thing that got me for a bit.. seems its current directory upon startup is / .
So my effort to test by writing to a file in the current directory made it look as though things were not working.
Use full path names in Daemon software.
If your site suddenly suffers a sharp drop in traffic, then you may be suffering from a Google penalty.
What is a Google penalty?
This is when Google, either via their algorithms or as a part of a manual review, drop your site from 15 to 950 places in their results. 30-60 places seem to be the most common.
There is a simple test to see if your site is being penalised. Search for just your domain name [ie: domain_name.com - without the .com] on Google, MSN, & Yahoo. If Google's position is significantly lower, then you most likely are suffering from one of these penalties.
Google does not officially acknowledge that they place sites under a penalty, but the SEO industry accepts that it does occur. You will notice that any public comment from Google with respect to penalties or changes to the indexing process is very carefully worded. Acknowledging that they treat sites differently could place them in a difficult situation if they were to be deemed a monopoloy [which lets face it, they clearly are.. 90% of my site's traffic comes from google, about 2% from the next search engine, Yahoo].
Why me? What's going on?
The simple answer is Google feels that your site breaches their webmaster guidelines.
The most likely culprits are:
large number of inbound links to your site suddenly appearing; probably as a result of links you've paid for.
large number of links from blog sites suddenly appearing; known as over promotion,
Your site has known blackhat techniques [Hidden text and links, 302 redirections instead of 301, etc..]
What can you do about ?
Getting angry will not help. Sending threatening legal letters will not help.
Google's anti-spam team, headed by Matt Cutts, are the ones responsible for your situation. They truly don't care about your difficulties.
<a bit of history>
Google have had & continue to have considerable problems with people gaming their index.
Their view is giving out information as to why sites have been penalised will help spammers. Thus, they generally don't respond to any requests.
They know they will cause people to suffer, but have made that decision.
They want to cost spammers as much time as possible.
</a bit of history>
Understand that Google now considers your site to be in breach of their guidelines, so find the cause & then fix it. Be prepared for this to take months, even over a year.
No, really. What can I do?
Firsly, sign up to Google's webmaster tools.
These are google's tools for webmasters to see how Google views their site. The dashboard also allows you to send a reconsideration request to google.
Before you send a reconsideration request, you need to identify what is wrong & fix it. Google will not tell you what is wrong, so its very much a random process.
1. Review Google's Webmaster tools & resolve any issues. Of particular interest should be:
- Diagnostics->Web crawl
- Diagnostics->Content analysis
- Links->Pages with external links
With respect to Links->Pages with external links. Review all the sites with links to your yours. If any of these are from paid advertising, Google may deem these to "paid links" & thus spam; so remove them from the offending sites ASAP.
If you have a large amount of inbound links from one site to yours, Google may deem this as suspicious.
If you have reciprocal links, again this pattern may be deemed as suspicious.
Do you sell links on your site? If so, then Google may not be considering you a spammer.
2. Review your site & fix the following:
- Validate your site with a html validator.
- Examine each page & look for any hidden text or links that that may appear to be
- If you have any redirections, ensure that they are 301 - moved permanently, as to many 302 redirections will look like blackhat techniques.
Once you have reviewed & fixed the above, then file a reconsideration request in Google's Webmaster Tools. Do not expect a response as you won't get one. Your traffic will come back all of a sudden, or not at all. If your traffic has not come back, then you know that you have missed something, so review your site, your site's outbound links, and inbound links. Once you feel that you have resolved the problem, file another reconsideration request.
I have read stories of people having their site penalised for over a year, its a very disheartening process.
sudo groupadd svn
sudo useradd -g svn svn
sudo svnadmin create /var/svn
sudo chown -R svn:svn /var/svn
svnadmin create /path/to/repository
#!/bin/sh -e
echo Starting Subversion
svnserve -d -r /home/matt/development/subversion_repository
Add new project
Checkout project
svn checkout svn://localhost/project
It ain't gonna happen.
VMware need to update their software to work with the latest linux kernel.
Arse :-(
Update: Use KVM instead.
I had a problem with installing Ubuntu on a Toshiba Tecra T9100 laptop. It got through most of the installation process, but then hung 15% of the way through detecting the target file system.
I spent a couple of days searching the web, finding similar issues & trying all sorts of boot options all to no avail.
The solution was quite simple in the end. This machine had 256MB of RAM, increasing this to 512MB fixed the problem & made the machine much more responsive.
I guess Ubuntu needs 512MB of RAM to be usable.
Snap have a very nice site preview feature. You can see it in action on this site with the little bubbles beside each link. It's not always up to date, but when it works, its great.
One slight problem, the javascript client is just not that bright. It's not AJAX aware, & if you update part of page, then the Snap script just ignores it: boo.
I've found a very simple way around this. Every time you replace a div via AJAX, make sure you run this script after the div has finished loading. This will tag the new url's with snap.
<script type="text/javascript">
SNAP_COM.shot_main_js_called = false;
SNAP_COM.shot_main_js();
</script>
The first command fools the snap script into thinking that is has not been run before, whilst the second calls the same function that's called on the window.onload event.
It really is a brain dead simple hack & I find it bit weird that this not a supported feature.
When first running Rails & Mysql, you might see thing like this in the log files.
WARNING: You're using the Ruby-based MySQL library that ships with Rails. This library is not suited for production. Please install the C-based MySQL library instead (gem install mysql).
*
You may think.. hello... I have installed the mysql gem via 'gem install mysql'. Indeed you may have, however this WARNING message means that rails is unable to load the required libraries.
To see what is going on, fire up the console & run the following commands.
require 'rubygems'
require 'mysql'
You may see something like this.
ruby script/console
>>require 'rubygems'
[]
>>require 'mysql'
LoadError: libmysqlclient.so.15: cannot open shared object file: No such file or directory - /usr/local/lib/ruby/gems/1.8/gems/mysql-2.7/lib/mysql.so
from /usr/local/lib/ruby/gems/1.8/gems/mysql-2.7/lib/mysql.so
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `require'
from ../../activesupport/lib/active_support/dependencies.rb:496:in `require'
from ../../activesupport/lib/active_support/dependencies.rb:342:in `new_constants_in'
from ../../activesupport/lib/active_support/dependencies.rb:496:in `require'
from (irb):3
So, Rails can see the libraries, but cannot load them.
Here is how I resolved the problem.
1. Uninstall the mysql gem - 'gem uninstall mysql'
2. Reinstall mysql like so.. 'gem install mysql -- --with-mysql-lib=/usr/lib/mysql'
3. Restarted mongrel & all was good.
Thanks Ezra for your many posts.
Notes for me as I keep forgetting..
New way of defining nested routes
map resources :posts, has_many => :comments
New nested resource form
<% form_for([@post, @comment]) do |f| %>
.. form code..
<% end %>
These are my notes for a short presentation on the recruitment side of team building - be they for short 6-9 week projects, for longer projects, and for full time employees to complement an existing team.
For me, team building simply is the act of putting together a group of people to accomplish a goal. The goal may be to get something done quickly, in a short period of time, in a high pressure environment, or it may be to do something reliably over a long period of time.
Team building is key component of recruitment, where you are not only looking for a skill-set, but also an individual to complement an existing team. Equally important to consider are the organisations and team strengths & weaknesses, the individuals skill set, their temperament, and their personal desires/motivations.
I have found it relatively easy to put together teams for short projects, where the goals are easily definable and the end-date is set. Even with quite disparate individuals. It is relatively easy to manage issues as all are aware that the environment is only for a short period and differences can usually be set aside for that period.
Longer term roles [ie: full-time roles or greater than 12 month projects] are a quite a bit more difficult. There is dichotomy of needing to fill the role, but also ensuring that new person will enhance the existing team. There will usually be plenty of suitably qualified applicants, deciding which one will best fit the organisationan and team needs is a challenge.
I try & put together a profile of the target individual in two parts. Firstly, the easy bit, profiling the technical knowledge & expertise required to perform the role. Secondly, the hard bit, what personal characteristics are desired in the target individual ; here we are looking for temperament, ability to handle change, ability to implement change in a group, willingness & ability to learn/contribute/argue.. etc. For full time roles this can take me a few weeks before I'm happy. For short-term, usually an hour or will suffice.
Technical competencies are very easy to review. Just ask the candidate a series of problem solving questions; what you are looking for is the approach to develop a solution, rather than the end result. Then ask the candidate why they have done things they way they have. I've learned quite a bit doing this - seeing several different approaches to the same problem that create the same solution.
I find the personal characteristics much more difficult to quantity. Having already thought through the desired characteristics should provide some guidance during the interview. If you can afford it, for long-term or full time positions, psychological testing can be of some value & help you round out a character or perhaps raise a flag where one was not apparent. I worked in very risk adverse organisations, a large bank & a large Insurance company & they both utilised IQ & psychological Myers-Briggs style testing for all candidates. I hated it at the time as it generally ruled out the most interesting candidates. Looking back know, I see that it was doing exactly what it was supposed to. A risk adverse organisation will value stability & predictability; rather than look for people who will naturally push boundaries in front of them & potentially cause friction.
For a startup or small to medium company you will mostly likely not be able to afford such testing, nor be able to afford significant chunks of your teams time to interview candidates. You will need to rely upon your experience, instincts & reference check previous employers & employees.
How a candidate will fit into & effect an existing team is difficult to predict. Not every organisation can be like Microsoft, Yahoo, & Google & fly candidates in from all over the world & have them spend large parts of a day with various groups. It is usually worthwhile to arrange for the existing team to spend at least sometime with the short-listed candidates, provide their feedback, & feel that they considered a valuable part of the process. I've never been 100% right with my predictions of how things would work out, sometimes pretty close, & a couple of time just dead wrong. There will always be some level of unpredictable result [ie: people changing their sexual orientation & having affairs.. who saw that one coming?].
At times I've weighed technical skills higher than team fit to challenge the status quo, knowing that some form of conflict would occur - a high risk strategy; it took months, but it paid off in the long run as it challenged the status quo had an effect upon the culture. At times I've weighed social skills higher than technical competence to help stimulate higher levels of communication. Change is always a high risk exercise & each recruiting decision is based upon a unique circumstances that have their own potential risks & rewards. Very rarely is the right answer staring right back at you.
I've made some fantastic decisions & some bad ones. What I have learned is that its best for all parties to recognise a bad decision quickly and take remedial action, whatever that be. I always start an employment relationship clearly articulating expectations and the process if things are going well, and the process if things are going bad.
Take aways.
- Understand what you are trying to achieve - organisational & team goals - be clear on your technical & personal goals for the role
- Use pschological testing if you can afford it, but also equally value your instincts & take responsibility for the decision.
- Get existing team feedback on short-listed candidates if possible.
- Reference check, reference check, reference check - employers & employers of the short-listed candidates
rake rails:freeze:edge # Lock to latest Edge Rails or a specific revision with REVISION=X (ex: REVISION=4021) or a tag with TAG=Y (ex: TAG=rel_1-1-0)