Archive for July, 2008

Use tcpdump for traffic analysis

Source: http://blogs.techrepublic.com.com/…1

Author: Chad Perrin

The tcpdump tool is an old mainstay of network debugging and security monitoring, and security experts all over the world swear by its usefulness. It is a command line tool that eschews all the makeup and jewelry of other traffic analysis tools such as Ettercap and Wireshark, both of which provide packet sniffing functionality with a convenient captive interface. In contrast to such tools, tcpdump takes a command at the shell, with options specified at that time, and dumps the results to standard output. This may seem primitive to some users, but it provides power and flexibility that isn’t available with the common captive interface alternatives.

Options

The tcpdump utility provides dozens of options, but I’ll just cover a few of them here:

  • -A: Print each packet in ASCII.
  • -c N: Where the letter N is a number, this option tells tcpdump to exit after N packets.
  • -i interface: Capture packets on the specified network interface.
  • -n: Don’t resolve addresses to names.
  • -q: Provide less verbose (”quiet”) output so output lines are shorter.
  • -r filename: Read packets from the specified file rather than a network interface. This is usually used after raw packets have been logged to a file with the -w option.
  • -t: Don’t print a timestamp on each line of output.
  • -v: Provide more verbose output. Verbosity can be increased more with -vv, and even more than that with -vvv.
  • -w filename: Write raw packets to the specified file.

Expressions

The tcpdump utility also supports command-line expressions, used to define filtering rules so that you get exactly the traffic you want to see, ignoring “uninteresting” packets. Expressions consist of a number of primitives and, optionally, modifier terms. The following primitives and modifiers do not constitute a comprehensive list, but they are among the most commonly useful.

Primitives

  • dst foo: Specify an address or hostname to limit captured packets to traffic sent to a particular host.
  • host foo: Specify an address or hostname to limit captured packets to traffic to and from a particular host.
  • net foo: Specify a network or network segment using CIDR notation to limit packet capture.
  • proto foo: Specify a protocol to limit captured packets to network traffic using that protocol.
  • src foo: Specify an address or hostname to limit captured packets to traffic sent by a particular host.

Modifiers

  • and: Use this to chain together primitives when you want to limit captured packets to those that meet the requirements of the expressions on both sides of the and.
  • not: Use this modifier just before a primitive when you want to limit captured packets to those that do not meet the requirements of the following expresssion.
  • or: Use this to chain together primitives when you want to limit captured packets to those that meet the requirements of one or more of the expressions on either side of the or.

Examples

All of these options and expression primitives and modifiers, along with others listed in the tcpdump manpage, can be used to construct very specific commands that produce very precise output.

  • tcpdump -c 50 dst foo can give you information that may help identify the source of heavy incoming traffic targeting an overloaded server with hostname “foo”, dumping the first 50 packets as output.
  • tcpdump -c 500 -w `date +"%Y%j%T"`.log dumps 500 packets to a file named with a current time/date stamp (e.g. 200820715:16:31.log) so that they can later be filtered according to the information you want to see. I have the command date +"%Y %j%T" aliased to stamp in my shell’s rc file, so I can shorten a command like this to tcpdump -c 500 -w `stamp`.log, saving me from having to remember all the formatting options for the date command off the top of my head.
  • tcpdump proto ssh src or dst foo and src and dst not bar produces ongoing output that shows all SSH activity originating from or targeting host “foo” unless it is originating from or targeting host “bar”. If foo is only supposed to be accessed via SSH by bar, this command will allow ongoing monitoring of unauthorized SSH traffic to and from foo. You could even start a number of persistent monitoring processes with tcpdump like this within a tmux session on a dedicated monitoring server.

As you can no doubt see, tcpdump’s expressions capabilities are roughly equivalent to a simple domain specific programming language that is extremely easy to understand. With that kind of power and flexibility at my fingertips, there’s little need to use anything else for general traffic analysis tasks.

  • Share/Bookmark

Comments off

Inside the Cloud: 9 Sectors to Watch

Source: http://gigaom.com/2008/07/20/9-cloud-computingsectors-to-watch/
Author: Alistair Croll

There’s already a ton of activity taking place in the cloud computing space, so much so that it can be hard to know who to watch. In many cases, it’s too early to pick winners. But there are distinct sectors of the IT industry that are particularly well suited to the on-demand, pay-as-you-go economics of cloud computing. Here are eight segments — and one company that’s a segment all its own — that we’re tracking closely.

Hosting companies that make the jump: When it comes to reliable managed hosting, Rackspace leads the pack. (Its VMware-based Mosso offering may appeal more to enterprises trying the cloud for the first time.) Clouds like XCalibre’s Flexiscale and Joyent are already there, but don’t have Rackspace’s installed base.

Stack-specific clouds: While Google and Amazon get the headlines, Engine Yard is heavily involved in the Ruby on Rails development community. Competitor Heroku is also Rails-focused, but relies on Amazon for its hosting platform.

Tools to wrangle virtual machines: To manage your EC2 machines, you’re going to need help. RightScale makes software for managing machines in the cloud; its tight focus on Amazon has made it an early favorite. Elastra, Enomalism and others have similar solutions.

Testing sandboxes: For many enterprises, a testing sandbox is the perfect way to start using on-demand infrastructure. CohesiveFT’s Skytap (a sister to Flexiscale) spins up testing machines in a cloud, but incumbent Surgient and recent entrant StackSafe aren’t far behind. And once you’ve tested a machine and seen that it works, why not leave it in the cloud?

Cloud-based development platforms: Companies like Rollbase and Coghead let non-developers build data-driven applications of any sort (as opposed to more specialized platforms like those of Salesforce and Ning.) But Intuit’s Quickbase, which now has access to Quickbooks data, has a head start: Millions of small businesses. Is this how SMB gets cloud?

Scaling frameworks: Wall Street needed fast, reliable applications that grew easily. Instead of adding more, bigger servers, they used Gigaspaces to bundle whole server clusters into discrete “processing units” that can be cloned to add capacity. In addition to being faster and scaling better, these units don’t care whether they’re in a private data center or a cloud.

Application delivery networks: What has tens of thousands of servers worldwide, a global network connecting them, and isn’t Google? Akamai. What was once a way of getting bits to far-flung corners of the Net is an often-overlooked cloud: Akamai has been able to run code at the edge since 2000. Its 2007 acquisition of Netli made it matter to enterprises even more. Akamai can weather heavy load and may be able to withstand attacks better than centralized clouds.

Cloud builders: 3Tera lets companies get into the cloud business. Enterprises can make in-house clouds on existing data centers; or service providerscan build their own cloud offeringsin the way Enki and others have. In 3Tera’s model, subscribers drag and drop the firewalls, servers and appliances they need. The company’s software then maps these virtual application stacks to servers and network segments. The results are impressive: On seeing 3Tera for the first time, ESM guru John Willis was so impressed he insisted on logging in to the icons on his screen to verify that it wasn’t just a demo.

The obvious one: Of the three big virtualization firms, only one (Microsoft) also has millions of desktops, two handset platforms, licensing for desktops, servers and applications, synchronization, and a huge online presence. Up until now, the Redmond giant has been treading carefully; it has to convert billions of dollars of shrink-wrap sales to on-demand revenue streams. But Microsoft’s going to be a huge player in the cloud.

  • Share/Bookmark

Comments off

A combination of Cloud, Web2.0 and Infrastructure

Facebook, Joyent and Sun have partnered to provide free scalable, on-demand infrastructure from Joyent to Facebook developers. Joyent’s Accelerator on-demand infrastructure (peered with Facebook’s datacenter!) provides the very best load balancers, routing and switching fabric, x86 servers and storage from Sun. Facebook developers can take advantage of Joyent Accelerators to quickly launch Facebook applications capable of scaling to millions of users. All for free.

What’s Included?

A Joyent Facebook Developer Accelerator includes everything you need to develop and deploy your Facebook application. You get root access to a virtualized machine that includes all the tools for developing Facebook applications in PHP, Rails, and Python.

Note: We do not currently support Windows, and do not plan to do so in the foreseeable future.

How it works?

  • Share/Bookmark

Comments off

Gartner: Seven cloud-computing security risks

Source: http://www.infoworld.com/…/Gartner_Seven_cloudcomputing_security_risks_1.html

Cloud computing is fraught with security risks, according to analyst firm Gartner. Smart customers will ask tough questions and consider getting a security assessment from a neutral third party before committing to a cloud vendor, Gartner says in a June report titled “Assessing the Security Risks of Cloud Computing.”

Cloud computing has “unique attributes that require risk assessment in areas such as data integrity, recovery, and privacy, and an evaluation of legal issues in areas such as e-discovery, regulatory compliance, and auditing,” Gartner says. (Compare security products.)

Amazon’s EC2 service and Google’s Google App Engine are examples of cloud computing, which Gartner defines as a type of computing in which “massively scalable IT-enabled capabilities are delivered ‘as a service’ to external customers using Internet technologies.”

[ Learn more about what cloud computing really means and the new breed of utility computing and platform-as-a-service offerings. ]

Customers must demand transparency, avoiding vendors that refuse to provide detailed information on security programs. Ask questions related to the qualifications of policy makers, architects, coders and operators; risk-control processes and technical mechanisms; and the level of testing that’s been done to verify that service and control processes are functioning as intended, and that vendors can identify unanticipated vulnerabilities.

Here are seven of the specific security issues Gartner says customers should raise with vendors before selecting a cloud vendor.

  1. Privileged user access. Sensitive data processed outside the enterprise brings with it an inherent level of risk, because outsourced services bypass the “physical, logical and personnel controls” IT shops exert over in-house programs. Get as much information as you can about the people who manage your data. “Ask providers to supply specific information on the hiring and oversight of privileged administrators, and the controls over their access,” Gartner says.

  2. Regulatory compliance. Customers are ultimately responsible for the security and integrity of their own data, even when it is held by a service provider. Traditional service providers are subjected to external audits and security certifications. Cloud computing providers who refuse to undergo this scrutiny are “signaling that customers can only use them for the most trivial functions,” according to Gartner.

  3. Data location. When you use the cloud, you probably won’t know exactly where your data is hosted. In fact, you might not even know what country it will be stored in. Ask providers if they will commit to storing and processing data in specific jurisdictions, and whether they will make a contractual commitment to obey local privacy requirements on behalf of their customers, Gartner advises.

  4. Data segregation. Data in the cloud is typically in a shared environment alongside data from other customers. Encryption is effective but isn’t a cure-all. “Find out what is done to segregate data at rest,” Gartner advises. The cloud provider should provide evidence that encryption schemes were designed and tested by experienced specialists. “Encryption accidents can make data totally unusable, and even normal encryption can complicate availability,” Gartner says.

  5. Recovery. Even if you don’t know where your data is, a cloud provider should tell you what will happen to your data and service in case of a disaster. “Any offering that does not replicate the data and application infrastructure across multiple sites is vulnerable to a total failure,” Gartner says. Ask your provider if it has “the ability to do a complete restoration, and how long it will take.”

  6. Investigative support. Investigating inappropriate or illegal activity may be impossible in cloud computing, Gartner warns. “Cloud services are especially difficult to investigate, because logging and data for multiple customers may be co-located and may also be spread across an ever-changing set of hosts and data centers. If you cannot get a contractual commitment to support specific forms of investigation, along with evidence that the vendor has already successfully supported such activities, then your only safe assumption is that investigation and discovery requests will be impossible.”

  7. Long-term viability. Ideally, your cloud computing provider will never go broke or get acquired and swallowed up by a larger company. But you must be sure your data will remain available even after such an event. “Ask potential providers how you would get your data back and if it would be in a format that you could import into a replacement application,” Gartner says.

  • Share/Bookmark

Comments off

Tag Database Schema

Source: http://www.pui.ch/phred/archives/2005/04/tags-database-schemas.html

mysqlicious sample datamysqlicious database stucture
In this solution, the schema has got just one table, it is denormalized.
This type is called “MySQLicious solution” because MySQLicious imports del.icio.us data into a table with this structure.

Intersection (AND)

Query for “search+webservice+semweb”:
SELECT *
FROM `delicious`
WHERE tags LIKE "%search%"
AND tags LIKE "%webservice%"
AND tags LIKE "%semweb%"

Union (OR)

Query for “search|webservice|semweb”:

SELECT *
FROM `delicious`
WHERE tags LIKE "%search%"
OR tags LIKE "%webservice%"
OR tags LIKE "%semweb%"

Minus

Query for “search+webservice-semweb”
SELECT *
FROM `delicious`
WHERE tags LIKE "%search%"
AND tags LIKE "%webservice%"
AND tags NOT LIKE "%semweb%"

  • Share/Bookmark

Comments off

3 Layers in Cloud

Source > http://refresh.gigaom.com/…defogging-cloud-computing-a-taxonomy/

  • Applications in the cloud (Salesforce and other SaaS vendors exist here today) provide turnkey end-user software, normally browser-based, with a specific functional focus. They are the easiest to start ‘consuming,’ but also the least flexible. They grow out of the ASP world of the late ‘90s and encompass the SaaS offerings of today.
  • Platforms in the cloud (Google’s AppEngine, Mosso, Heroku are good examples) offer turnkey environments into which a developer can plug in code written within certain guidelines or restrictions (programming language, data-store model, etc.), and scaling is performed “behind the curtains” by the platform.
  • Infrastructure in the cloud (Amazon Web Services, Flexiscale, and others) is the most flexible offering, providing compute and storage resources in a primitive, close-to-bare-metal API interface, that can be leveraged in a multitude of ways with few restrictions – but which also require more up-front work to design and implement. This is where our company RightScale focuses – we offer a cloud management platform for low-level ‘infrastructure in the cloud’ resources that preserves flexibility and power, while offering quick deployment and easy management.
  • Share/Bookmark

Comments off