Data scraping tool for non-coding journalists launches

A tool which helps non-coding journalists scrape data from websites has launched in public beta today.

Import.io lets you extract data from any website into a spreadsheet simply by mousing over a few rows of information.

Until now import.io, which we reported on back in April, has been available in private developer preview and has been Windows only. It is now also available for Mac and is open to all.

Although import.io plans to charge for some services at a later date, there will always be a free option.

The London-based start-up is trying to solve the problem of the fact that there is "lots of data on the web, but it's difficult to get at", Andrew Fogg, founder of import.io, said in a webinar last week.

Those with the know-how can write a scraper or use an API to get at data, Fogg said. "But imagine if you could turn any website into a spreadsheet or API."

Uses for journalists

Journalists can find stories in data. For example, if I wanted to do a story on the type of journalism jobs being advertised and the salaries offered, I could research this by looking at various websites which advertise journalism jobs.

If I were to gather the data from four different jobs boards and enter the information manually into a spreadsheet it would take would take hours if not days; if I were to write a screen scraper for each of the sites it would require knowledge and would probably take a couple of hours. Using import.io I can create a single dataset from multiple sources in a few minutes.

I can then search and sort the dataset and find out different facts, such as how many unpaid internships are advertised, or how many editors are currently being sought.

How it works

When you download the import.io application you see a web browser. This browser allows you to enter a URL for any site you want to scrape data from.

To take the example of the jobs board, this is structured data, with the job role, description and salaries displayed.

The first step is to set up 'connectors' and to do this you need to teach the system where the data is on the page. This is done by hitting a 'record' button on the right of the browser window and mousing over a few examples, in this case advertised jobs. You then click 'train rows'.

Building an extractor using import.io

Building an extractor in import.io

It takes between two and five examples to teach import.io where all of the rows are, Fogg explained in the webinar.

The next step is to declare the type of data and add column names. For example there may be columns for 'job title', 'job description' and 'salary'. Data is then extracted into the table below the browser window.

Data from different websites can then be "mixed" into a single searchable database.

Dataset page import.io

The dataset page

In the example used in the webinar, Fogg demonstrated how import.io could take data relating to rucksacks for sale on a shopping website. The tool can learn the "extraction pattern", Fogg explained, and apply that to to another product. So rather than mousing over the different rows of sleeping bags advertised, for example, import.io was automatically able to detect where the price and product details were on the page as it had learnt the structure from how the rucksacks were organised. The really smart bit is that the data from all products can then be automatically scraped and pulled into the spreadsheet. You can then search 'shoes' and find the data has already been pulled into your database.

When a site changes its code a screen scraper would become ineffective. Import.io has a "resilience to change", Fogg said. It runs tests twice a day and users get notified of any changes and can retrain a connector.

It is worth noting that a site that has been scraped will be able to detect that import.io has extracted the data as it will appear in the source site's web logs.

Case studies

A few organisations have already used import.io for data extraction. Fogg outlined three.

    British Red Cross

The British Red Cross wanted to create an iPhone app with data from the NHS Choices website. The NHS wanted the charity to use the data but the health site does not have an API.

By using import.io, data was scraped from the NHS site. The app is now in the iTunes store and users can use it to enter a postcode to find hospital information based on the data from the NHS site.

"It allowed them to build an API for a website where there wasn't one," Fogg said.

    Hewlett Packard

Fogg explained that Hewlett Packard wanted to monitor the prices of its laptops on retailers' websites.

They used import.io to scrape the data from the various sites and were able monitor the prices at which the laptops were being sold in real-time.

    Recruitment site

A US recruitment firm wanted to set up a system so that when any job vacancy appeared on a competitor's website, they could extract the details and push that into their Salesforce software. The initial solution was to write scrapers, Fogg said, but this was costly and in the end they gave up. Instead they used import.io to scrape the sites and collate the data.

Source:http://www.journalism.co.uk/news/data-scraping-tool-for-non-coding-journalists-launches/s2/a554002/

7 Strategies for Writing Product Descriptions that Sell

Creating compelling product descriptions that encourage consumers to purchase is one of the most important content creation tasks. Perfecting your skill in this area can open up new opportunities for you as a writer and can allow you to leave your mark in the retail world.

These short bites of text provide solid information for consumers and promote the product lines of online retail stores, making them a good source of income for talented writers. Here are seven tips to polish your skills in the product description field.

    Highlight One Important Feature
    Address Consumer Concerns
    Personalize Your Approach
    Keep It Short and to the Point
    Stick to the Facts
    Write for Your Target Audience
    Maintain a Consistent Tone

1. Highlight One Important Feature

Rather than simply reciting a list of the general attributes of a commonly sold item, focus on one unique feature, attribute or use of the item. For example, an everyday No. 2 pencil in a cheerful shade of yellow could be described in this way:

Fill in those standardized test papers with a little help from our sunny No. 2 pencil. This writing instrument is ideal for everyday use and features a handy built-in eraser to correct any mistakes quickly and easily.

This approach is less useful for complex items with numerous distinguishing characteristics. However, it can provide a solid basis for short blurbs on common household or office items.

2. Address Consumer Concerns

One proven way to create compelling product descriptions is to put yourself in the buyer's shoes. What would you want to know about the item?

This is especially important for large-ticket items that constitute a major investment for most consumers. For automobile purchases, fuel economy and performance may be the primary concerns.

A description of a towel rack should include information about the material used and the surface finish to help consumers determine whether the item will match their existing decor and will stand up to regular use.

By considering the likely concerns of buyers and addressing them in your description, you can produce better results for your retail content customers.

3. Personalize Your Approach

A cold, clinical description can turn off your readers and result in fewer sales. By maintaining a somewhat more informal tone and adding humor when appropriate, you can boost the appeal of your product descriptions and create a more conducive mood for purchasing decisions.

Consumers will respond better to a warm, friendly tone in most cases and will reward your efforts with increased sales.

4. Keep It Short and to the Point

Consumers in the online world tend to have short attention spans. Your descriptions should hit the high points and provide adequate information on which to base a purchase decision. However, there's typically no need to include a history of the product in the consumer marketplace or a loving description of the process by which the item was made.

Stick to the point and deliver the information you would want and need as a consumer to provide the greatest benefit for your clients in the retail marketplace.

5. Stick to the Facts

Product descriptions are not the place for flights of fancy or false information. By providing factual descriptions with no misleading embellishments, you can ensure that consumers get the necessary data to make a purchase while ensuring that your representation of the product is accurate and fact-based.

This can protect your corporate clients from legal entanglements and ensure continued demand for your content creation services in the product description field.

6. Write for Your Target Audience

Doing a little market research on the demographic groups most likely to buy your chosen item can help you to hone your writing skills and produce exceptional copy for product descriptions. By pinpointing your target audience, you can increase their engagement with the product and win more sales for your clients.

A minivan, for instance, is generally purchased by families looking for extra room for passengers and plenty of cargo space; writing your product description with those concerns in mind can significantly improve the results of your work.

7. Maintain a Consistent Tone

While you may have developed a unique voice in your writing endeavors, product descriptions require that you adopt the client's tone as your own for best results. By reading over the site and paying careful attention to other product descriptions or online copy, you can create product descriptions that fit into your client's online catalog seamlessly and naturally.

This can increase the value of your writing to your client and can provide a more organic experience for consumers who visit the site.

Closing Thoughts

By incorporating these strategies into your writing process, you can create compelling and persuasive product descriptions for your clients and help them to increase their sales in the online marketplace. This can ensure repeat business from your established clients and help you to build a reputation for excellence in the content creation field.

Additional Resources

For more information, check out the resources below for the aspiring product description writer.

Source:https://www.crowdcontent.com/writer-resources/product-descriptions

Scraping software, services and plugins sum up

Since we have already reviewed classic web harvesting software, we want to sum up some other scraping services and crawlers, scrape plugins and other scrape related tools.

Web scraping is a sphere that can be applied to a vast variety of fields, and in turn it can require other technologies to be involved. SEO needs scrape. Proxying is one of the methods which can help you to stay masked while doing much web data extraction. Crawling is another sub-technology indispensable in scrape for unordered information sources. Data refining follows the scrape, so as to deal with the unavoidable inconsistency of harvested data.

In addition, we will consider fast scrape tools, making our life better, and some services and handy scrapers which enable us to obtain freshly extracted data or images.

Web Scraping directory (classified by function)

Crawling
   
Proxy for scrape
   
Scrape services

Scrape plugins
   
Anti-scrape service
   
Tracking for change

Scrape for SEO
   
Fast scrape
   
Handy scrape
   
Scrape legal issues
   
Fast Scrape

Often I need to get something fast from the screen into my pocket. How to do it without invoking web scraping applications? What can help me?

Scraper, the Google Chrome extension is what makes my life easy. I’ve installed this extension in my favorite browser

( :-) ) and have this tool always embedded in the right-button menu. I highlight the sample area and right-click, and the same page area content is on the display, and with the next click, the content is on a Google spreadsheet. It is as easy as possible: no applications to run, no data samples, no target folders and other such things.

Another data extraction tool available, called TheWebMiner,  is one in the cloud. This cloud scraper lets you just manually enter data samples from the target site, and it will automatically define similar data and harvest them. The result is downloadable in CSV, XML and JSON formats.

Scrape services and tools

Among the scrape services we take note of:

Grepsr scraping service. This service allows administrators to set up a scrape project but still be able to control the scrape scheduling and other data extraction steps.

Inspyder, the application for scrape and crawl. It’s good for crawling first as many pages as possible, and then scraping by applying a predefined pattern.

The A1 Website scraper works to extract text, URLs etc., using only Regexes. The output is saved into a CSV file. This scraper allows multifaceted tuning for web scraping. However, in mass data gathering, it consumes a lot of time.

Anti-scrape service

Since web scraping methods are being commonly used, many are concerned with malicious scrapers stealing website data, mirroring proprietary databases or throttling a site’s bandwidth. Why not have some protection against these invasions? We’ve reviewed an anti-scrape service, called Distil, that proved to be very robust and trustworthy. What I liked about it is that among other anti-scrape services, it’s quite user friendly.

Crawling tools

Then there are cases when users or companies do not need to get much data from the web, but rather they just need to crawl some web pages and index them based on certain criteria. What tools can help here? How about the 80legs service that does web crawling utilizing the power of thousands of widely distributed consumers’ computers while they are in idle mode? The claimed crawling speed is one to be ranked with modern search engines.

Scrape plugins

Need to acquire some fluctuating data to insert into your Word Press driven web page? The Web Scraper Shortcode plugin is good for that. Just insert it into the html code with the specified URL and desired element notation, and your page gets enriched with the elements of the extracted pages with set limits.

Another geeky tool is the WP Web Scraper, the Word Press plugin that works to extract web data into custom Word Press pages. The scraper uses a cURL extraction library for scraping and phpQuery for parsing HTML. This tool is a highly flexible plugin having plenty of the optional arguments: Regex replacement, basehref adding to the links, cache data timeout, target page decode and others.

Scrape for SEO

How can scrape help your website’s SEO? To fix the broken links to your website requires identifying them. In the video of SEOMoz you can watch how to do it and also find out more about XPath and Regex techniques. The link to the simple Twitter scraper is available there as a bonus.

Sometimes you need to gather together all your blog’s posts as they are indexed by Google. How to do a custom Google search results scraper (based on Outwit Hub) is really interesting to watch in this video.

Tracking a webpage for changes on it

Web scraping is often needed in conjunction with tracking particular info. Why harvest the whole content if no or only tiny changes occurred? In this case you do not need to scrape the page but rather only be aware of some changes on the monitored sites. These kinds of tools, keeping track of target page changes, both free and paid are reviewed at this post: Web Page Change Tracking.

For how to apply one of the free change tracking tools to a particular target page, you can go to this post.

Proxy for scrape

How do I set up my own scraper with proxy without programming or sophisticated proxy services sign up and tune up? The ScraperWiki is a toolset and a platform that makes this possible. This free service allows you to load and run any scraper written on PHP, Python or Ruby. Yes, its original purpose is to let people write or adopt a scraper for non-profit data gathering, but, in my experience, I’ve run my custom scraper on ScraperWiki for the sake of proxying.

Handy scrape

Why spend extra time and effort to visit the same page just to monitor tiny elements? If you want to look over a picture of the week or news of the day, use the Handy Web Extractor comfortably residing in your PC tray. This tiny handy tool will make life easier for you, emancipating you from the daily opening of the same pages.

Scrape legal issues

The legal issues concerning scrape or employee monitoring have always been an important consideration and worthy of careful attention for most lawful web users. So we call to your attention two posts: How to alarm if your website is under illegal scrape and Ethical issues of using employee monitoring software.

Summary

Web scraping, web mining, data extraction and website scrape encompass indeed a wide range of application technology. In spite of some malicious use of them, web data scraping serves well for business intelligence in the following areas (but not limited to these):

    web crawling services
    data scrape services
    seo improvement
    changes tracking
    fast scrape

The adjacent area of the web scraping is the website changes tracking and monitoring.

Source:http://scraping.pro/scraping-software-services-and-plugins-sum-up/

Document scanning taken to the next level

Managing a whole firm is work. Whether it be little, moderate or big - to be the head of procedures may be sometimes overwhelming. It is usually smart to use outsourcing for a few of the administration crucial areas to other businesses. It is possible to outsource statigic planning, advertising and even data processing. There has been other intriguing outsourcing techniques experiments through the years but none of them succeeded. One break through of outsourcing that is gathering popularity these days is document scanning services. This type of service will manage your whole databases of documents. You don’t need full containers of documents available anymore.

This document scanning company is concentrated just on document scanning service. Whatever happens to your head office - you may rest assured that your valuable paperwork are kept and protected in an additional place. There've came out a number of document scanning companies on the market. Many are clearly better than others. Even so, one of the better and the most well-known document scanning firms is recognized as Saxon Archives. These men have been on the marketplace since 1995 and are at present serving hundreds of companies. This document shredding company is ranked one of the better in the field. The company is more than pleased.

In case you are fired up to find out more regarding Saxon Archives then you should go to their formal online site at the following website saxonarchives.com. Here is the best document shredding florida business for your use. You'll be able to get rid of all the undesired documents in an instant. Should you be looking for a first class, secure and straight forward florida document shredding answer - this is the greatest one. The mobile shredding choice provides a grin to your face. All your difficulties can go away with simply one phone or a touch in an software on the cell phone.

The mobile document shredding alternatives have been getting the marketplace quite some time but no-one can really say their level is the same as of those guys. The Saxon Archives’ document scanning business services are rated the best on the internet. You can check out the critiques and testimonies - they talk on their own in regards to the good quality and the exceptional assistance. Also, in case you are thinking about creating a data source from office document scanning paper then these kinds of gentlemen will also help you with all the daunting job. They'll digitalise all you report database.

Source:http://coltsdigital.me/development/index.php?do=/blog/76937/document-scanning-taken-to-the-next-level/

Foursquare Data Scraping

Benefits Of Foursquare Data Scraping Finding out information about people and potential clients is a great tool for a business.

The problem is many people are not sure about how to get this data and not even aware of the benefits of using foursquare data scraping to help them out. Once the businesses are aware of these benefits, they will not want to use anything else to help them out in their market research. The first benefit the businesses can find with this type of work is they are able to get the information for a low cost compared to other market research. So the business can find out more information about the customers, but not have to be concerned about paying a fortune to get all the studies done. Without this, the company may have to do the long and drawn out research studies which they do not want to do because of the cost.

Another benefit the company can find is this type of work does not take long to complete. Completion of this type of research because of how it is completed on the computer does not take long to get the results people want. Since they are getting these results in a timely manner, they do not have to be concerned about their research taking years to complete, leading to a potential loss of revenue or customers. Something else is this can help establish a profile of the customers who are coming to their business and what other likes they have. Typically people check into various places using this program.

By knowing what other shops people are checking into the business can determine where the customers are going and what other stores they are visiting. Having information on the customers can be a great thing to have. However, some people need to have some knowledge on the benefits of using the foursquare data scraping. By knowing about the benefits these can provide in market research for a company, they will see this as a great option. Then the company will not want to look for any other form of market research to help keep them competitive.

Source:http://thewebscraping.com/foursquare-data-scraping/

How Linkedin Profile Scraping Can Help

Your Business As the world changes and technology evolves, there has been an increased demand in automation tools. These tools help to reduce the amount of labor personnel that businesses need to hire. What used to take a long time is now done in a couple of minutes when automated tools are used. Linkedin Profile Scraping services allow businesses to get a huge list of contacts in just a matter of minutes. These services cut the burden on employees and increase the speed at which the job is done. These services allow any business to quickly and effectively collect information without having to hire someone to sit there and do it manually.

Every business needs a long list of contacts if it wants to be successful. It is impossible for a business to succeed without contacts in the industry. The problem is that this contact information is often hard to come by. When someone does happen to have this information, they are not willing to get rid of it at a reasonable price. Linkedin Profile Scraping companies make this process a lot easier. These companies specialize in scraping up to date data at affordable prices. They have specially designed software that can segment and search through all the databases of people in Linkedin. They can offer such a great discount because their software can do the job with minimal supervision. Once the software is created, there is not a lot of overhead for these companies. They then pass the savings on to their customers.

When you get your project report, the information will be available in an easy to sort text or csv file. This simplified information will only contain the fields that you asked for. This is typically a person’s name, phone number and email address. You can ask for whatever information is relevant to your needs and the company can scrape it for you. Linkedin Profile Scraping companies offer a simple solution to a problem that has plagued businesses for years. They offer a simple alternative to the usual ways of obtaining contacts. You can easily amass a large amount of contacts when using one of these companies.

Source: http://thewebscraping.com/linkedin-profile-scraping/