Sometimes you need to automate some terminal commands within your web application. I personally prefer PHP over other server side languages, and in this case its ability to run such commands are fantastinominal. There are a bunch of built in functions for securing/escaping arguments and commands, and a bunch of methods for executing shell commands. The main differences between them are the way output is returned to php. For most cases you should be fine using escapeshellarg() and shell_exec() methods - assuming you're using variables posted to your server code as arguments. You should read up on the various program execution methods over at php.net, and research all the implications and security risks involved in using them.

This post doesn't focus on their use, but instead on how to give Apache(or whatever web server you're using) root access on your server. In fact what you need to do in order to simplify your scripts is allow the Apache process to run root commands without a password. That's RIGHT, without a password. This can be exceptionally dangerous so you may want to limit this root access specifically the no-password-necessary root access only to specific programs you need to run from your scripts. Otherwise a small programming error would let malicious people take full control of your web server with ease.

The main purpose of enabling no password root access here is so you can easily run programs with a single command and not worry about being challenged for a password or having to deal with that in your server code. It's potentially more dangerous to store your root password in a public facing script than giving it no-password-root-access to a single program. A fair amount of web software and tools will have versions of their commands that can be run on a single line for this purpose.

This is for Ubuntu, but should work on most other distros with little tweaking.

First add the following line to your php script:

echo shell_exec("whoami");

This will output the user that Apache, or whatever server is running your php file, is running as on the system. Typically Apache runs as www-data, but your system may be set up differently.

Now open a terminal and ssh into your web server. Run the following command:

sudo visudo

What this does is edit the /etc/sudoers file, however using the visudo command is necessary for changes to properly take effect. Go to the bottom and add the following line to enable the Apache user to sudo without a password:

www-data ALL=NOPASSWD: ALL

The first ALL refers to hosts, the second ALL refers to programs/commands. If you only want to grant Apache sudo access to a specific program replace the second ALL with the full path to the command file. So even though you will be able to call last from your script without worrying about the path, you need to know the actual path here:

www-data ALL=NOPASSWD: /usr/bin/last

Now you should have a list of shortcuts at the bottom of the terminal, you want to "WriteOut"(ctrl+o) the file which is the same as saving it, you'll be prompted to choose the path to save to, make sure that you're saving it as /etc/sudoers, otherwise it may try save your changes as a copy.

You can now try run last from your php script by adding the following to your php script:

echo shell_exec("sudo last");

Now that it works you may want to remove the echo lines from your script, or test it with a different command since showing the world who's actually running Apache or the output of last is not something you want.

I needed a function to strip out html tags from a text input, but still let me specify allowable tags.

Instead of spending time figuring out the regular expressions needed to pull it off and becoming a better programmer, I figured why repeat work someone else has probably already done.. I mean I could be a busy man. Anyway I found this great function on Flexer.info [link]. But after trying it out I noticed that the one tag I really really wanted to be parsed out iframe wasn't. It seems because I had specified i as an allowable tag it was also accepting iframe.

So with all due respect to Andrei, below is the revised function with the security hole patched.

All I changed was near the bottom where it determines if it's an allowable tag or not the reg exp was

<\/?" + tagsToKeep[j] + "[^<>]*?>

which allowed any character to follow the allowed tag as long as it wasn't a nested tag, which included frame following i. This will also support self closing tags.

 
// strips htmltags
// @param html - string to parse
// @param tags - tags to ignore
public static function stripHtmlTags(html:String, tags:String = ""):String
{
    var tagsToBeKept:Array = new Array();
    if (tags.length > 0)
        tagsToBeKept = tags.split(new RegExp("\\s*,\\s*"));
 
    var tagsToKeep:Array = new Array();
    for (var i:int = 0; i < tagsToBeKept.length; i++)
    {
        if (tagsToBeKept[i] != null && tagsToBeKept[i] != "")
            tagsToKeep.push(tagsToBeKept[i]);
    }
 
    var toBeRemoved:Array = new Array();
    var tagRegExp:RegExp = new RegExp("<([^>\\s]+)(\\s[^>]+)*>", "g");
 
    var foundedStrings:Array = html.match(tagRegExp);
    for (i = 0; i < foundedStrings.length; i++)
    {
        var tagFlag:Boolean = false;
        if (tagsToKeep != null)
        {
            for (var j:int = 0; j < tagsToKeep.length; j++)
            {
                var tmpRegExp:RegExp = new RegExp("<\/?" + tagsToKeep[j] + " ?/?>", "i");
                var tmpStr:String = foundedStrings[i] as String;
                if (tmpStr.search(tmpRegExp) != -1)
                    tagFlag = true;
            }
        }
        if (!tagFlag)
            toBeRemoved.push(foundedStrings[i]);
    }
    for (i = 0; i < toBeRemoved.length; i++)
    {
        var tmpRE:RegExp = new RegExp("([\+\*\$\/])","g");
        var tmpRemRE:RegExp = new RegExp((toBeRemoved[i] as String).replace(tmpRE, "\\$1"),"g");
        html = html.replace(tmpRemRE, "");
    }
    return html;
}
 

http://www.paulgraham.com/gh.html

Visit the above link, a brilliant essay on productivity derived from programming.

It's strange thinking of how drastically my life has adapted to writing software over the years, especially the last year, compared to people I know who don't[read: everyone I know]. But this guy nails it. I guess there's something about programming that just moulds you into a certain behaviour set. It's like if I broke a chunk off Canada, floated it out into the middle of the Atlantic and started a civilization - leaving it for a few generations. Then mailed them an essay on small island civilizations they'd be like "Oh shit! Hey guys, someone wrote an essay about us," and you wouldn't actually have to visit the island to write about it.

What you should take from the essay is that it's all about context. The more context you can fit into your head about a given anything, the better you are at innovating. And when the technological advantage is even - meaning using a computer vs. using a typewriter, brilliance can only be judged on context.

via @AndrewWarner via news.ycombinator, via paulgraham

I'm writing a Flex/AIR app that grabs stuff from a database and displays it in an mx.controls.list. Interacting with it you can switch the list mode, which changes the visibility of certain controls in the itemRenderer. There are currently 20 items in the dataProvider, about 8 are displayed at any given time. I noticed that switching modes - and by doing so waiting for validateList() to run, took incrementally larger amounts of time for each of the first 3 items that were in view. So if you scrolled down one item and switched modes it was a bit faster, and if you scrolled past the first 3 switching modes became instant as it should be.

After looking over the same possibly relevant lines of code several times, reading up in detail of how the validateList() cycle works and getting into the nitty gritty of list classes I realized the problem was on the database side. I had a column named length. At first I thought there was an issue where I'd set the column type as a floating point number and maybe actionscript was having a time converting it or dealing with it in an object - there's no actual reason why I thought this, but the performance issue was not noticeable if the floating point number was smaller than 10,000.

Fortunately after only a few hours time wasted I, the spaz writing this, realized that the length column was being interpreted in actionscript as the length(ie: number of children/values) of the object. So say length was set to 100,000, for every item in the list it would have to create and analyze 999,992 blank values - creating space in memory for each one, along with the 8 actual values pulled from the database.

Furthermore when I referenced the item.length value while technically the value pulled from the database, was really the number of children in the object. The small robots that live inside my computer and make it work must have though I was bananas.

I'd like this to be my formal application for the prestigious Leader of the Idiots, but since I'm obviously not equipped with the basic skill set to do anything(read: dressing oneself, remembering reserved names) I'll rely on some kind soul reading this to file the application for me and submit it to the proper authorities.. thanks.

Adobe is literally making it possible for me to create what I want, and cosmically in sync with where I am on the road to creating those things. I feel indebted - as I should, to the tool creators. The future will be a crazy cross platform interconnected kinda world. It's a great era to live in, and be a part of.

Watching some political stuff on Youtube and saw the ad on the side was for Google. It's rare you see Google marketing their stuff, but it seems when they do they tend to use their own platforms to do it. And why not they have access to billions of pageviews and they already know that people using Google like Google.. As well as tons of other info about you from using their services and interacting with Adsense all over the web.

Search Stories is a YouTube Channel that features the Google search bar. There are already a bunch of episodes and you watch a 60 second plot unfold through a series of searches. They built a custom episode player with sleek effects and if you catch it as it's loading you'll see the default Flex busy mouse cursor. Google uses Flex, I'm pretty sure they use it for the YouTube player as well and some of their other Flash constructions but I wonder how it ties in to their relationship with Adobe. Since they offer API access and libraries in Flash and Flex you'd think they'd already have had to develop their own framework for working in Flash.. Then again maybe they just use the best tool for the job.

In Google's words, more videos after the jump.

(more...)

The Adobe Air 2.0 beta is available for download now. Air 2.0 is a huge advancement and brings tons of impactfull new features. Most notably you can now write fully native apps, that use fully native installers and communicate and launch native apps. Hugely improved performance. Work with sound like never before. Sockets, and HTML5.

Download the AIR 2.0 Beta.

The Flash 10.1 prerelease is also available for download.

This is from the Adobe Labs.

New Features in AIR 2

• Open documents with the user's default application
  You can open document files in the application the user has set up as the default opening application. Launching executables or script files is not permitted unless the application is packaged in a native installer. See File.openWithDefaultApplication() and Opening files with the default system application.
• Microphone data access
  You can access the sound data from a microphone directly for recording or other processing. See Microphone.sampleData and Capturing microphone sound data.
• Mass storage device detection
  You can now detect when a mass storage device, such as a USB drive or a camera, has been connected to the user's computer. See StorageVolumeInfo.
• Updated WebKit version
  • WebKit in AIR is based on the version shipped with Safari 4.0.3
  • Support for JavaScript profiling
  • SquirrelFish Extreme JavaScript engine resulting in 50% faster performance using SunSpider tests
  • CSS3 Module support (2D transformations, transitions, animations, gradients, zoom and WebKit CSS selectors, etc.).
  • Styling scrollbars via CSS
  • Latest Canvas enhancements.
• Global Error Handling
  Global error handling lets you handle all uncaught errors (both synchronous errors and asynchronous error events) in one place in your code. See UncaughtErrorEvent.
• New networking support
  • Server sockets
    You can listen for incoming socket connections. See ServerSocket and Server sockets.
  • UDP sockets
    You can send and receive messages using the Universal Datagram Protocol (UDP). See DatagramSocket and UDP sockets (AIR).
  • TLS/SSL sockets
    You now connect to a server that requires TLSv1 or SSLv3 for socket communications. See SecureSocket and Secure client sockets (AIR).
  • DNS lookup
    You can look up Domain Name System (DNS) resource records. See DNSResolver and Domain Name System (DNS) records.
  • Network interface enumeration
    You can enumerate the list of hardware and software network interfaces available on a client computer. See NetworkInfo and Network interfaces.
• Packaging an AIR application in a native installer
  You can package an AIR application in a native install program instead of an AIR file. Applications packaged and installed using a native installer have access to platform-specific features such as the Native Process API. See Packaging an AIR application in a native installer.
• Native process API
  You can launch and communicate with native processes. Applications that use this API must be installed using a native installer. See NativeProcess and Communicating with native processes in AIR and the following quick start articles: (For HTML developers) Interacting with a native process, (For Flex developers) Interacting with a native process, and (For Flash developers) Interacting with a native process.
• Database transaction savepoints
  You can commit or roll back multiple database changes as a single transaction. See SQLConnection.setSavePoint().
• Screen reader support (Windows only)
  Users can use screen reader software with AIR applications. See Accessibility.
• Printing enhancements
  • Vector printing is now supported on the Mac.
  • PrintJobOptions.printMethod allows you to control whether vector or bitmap printing is performed. You can also let the runtime choose the best method (using an internal heuristic based on analyzing the content to be printed). See PrintJobOptions.printMethod.
• Idle time-out settings for URL requests
  You can override the default idle time-out setting for URL requests. See idleTimeout.
• Improved IPv6 support
  IPv6 format addresses can now be used with all APIs that accept an IP string as input.
• Increased maximum size of NativeWindow
  The maximum size of a NativeWindow has been increased to 4095x4095 pixels (from 2880x2880 pixels). See NativeWindow.bounds.
• File promises (Win/Mac only)
  You can now allow users to drag a file that has not been created yet out of an AIR application and provide the data for that function after the file promise has been dropped. For example, you could provide a list of files available on a remote server. When the user drops an item from that list onto their desktop, you can download the data and write it to the dropped file. See URLFilePromise and Dropping file promises.
• Multi-touch events
  You can listen for multi-touch and gesture events on computers that have multi-touch hardware and operating system support. See TouchEvent.
• IME API and IME text input enhancement
  Input Method Editors (IMEs) can be used with any InteractiveObject, not just TextField objects. This allows you to support IME input when using the Flash Text Engine or the Adobe Text Layout Framework, for example. See IME.
• Native runtime install packages for Linux
  Native install packages are now provided for the rpm and debian package managers.

A flaw was discovered in the tls renegotiation process where a 'man in the middle' could take over the connection in a number of ways and perform a number of exploits. Transferring login, credit card, and other important info using https:// is no longer considered secure. The 'man in the middle' could be bumming off your local wireless network, anywhere in between you and your ISP, your ISP and the destination, or on the destinations network. Secure Certificates from the likes of VeriSign can no longer be fully trusted until they find a fix, at which point you'll need to update all your software - browsers, email clients, twitter apps, smart phone firmware, as all the software manufacturers implement and roll out the new(not yet figured out or released) protocol.

Luckily the vast majority of internet users are stupid and this won't affect activity on the internet a bit, even people that are reading or writing this post will still log into their email accounts and go about their online life relying on sheer improbability of them being exploited by this massive(read: catastrophic) security hole. Once again it's stupidity and recklessness that will keep the world turning cause as we all know if you stop and think about anything too long you'll just give up and go live in the forest like we were originally supposed to.

If you're interested in a more technical description go here.

If you're a programmer and you contribute to or write software which implements tls please disable renegotiation a.s.a.p. and push the update to all your users until a new version of the protocol is released.

via Ars Technica

First of all GRRRRRR!!

Second, this has been one of those things I randomly get sucked into between projects where I'll spend 5 hours on Google trying to figure it out and getting tiny fragments of info but never actually solving the issue. This is the worst! What the hell am I talking about? Say you use MAMP or whatever as a local testing server. You write some PHP and you need to use the mail() function. You test your new email function to your personal gmail account. Ok so you try it and it doesn't work, or even worse it works a couple times and then never again.  So you go to Applications->Utilities and fire up the Console application. You're shocked to see that there's a message in there saying something about Gmail not accepting mail from your IP address because it's registered as a residential thingy and apparently a lot of spammers use their personal computers to send spam.

So you say no no there must be some mistake I'm a programmer, not a spammer, I'm just trying to test out my new app. But you quickly realize you're talking to a computer, pleading, and well it doesn't care. typical. After, you cry and try piece together a coherent step by step set of instructions to route all mail sent from your computer through your Gmail account - so it would be from you, and all go through. here's what you do.

note that $ is used to show a new terminal command, you don't actually type it in:

Open Terminal - found in Applications->Utilities and type in:

$ sudo nano /etc/postfix/relay_password

You'll now be editing a new file called relay_password in the nano Terminal editor, type in the following substituting your login info - it should work with google apps accounts as well:

smtp.gmail.com example@yourdomain.com:yourpassword

Press ctrl+o on your keyboard followed by Enter to save the file, then press ctrl+x to exit the editor.

Now type in:

$ sudo postmap /etc/postfix/relay_password

That should tell Postfix to use the relay file you just created. Gmail uses a secure connection so you need to head over to Verisign and download some root certificates. Go to the following url fill out your info and download the .zip file:

https://www.verisign.com/support/roots.html.

Now type in the following commands one after another. In the second command it wants the roots.zip you just downloaded, you can just drag the zip file onto the Terminal window and it will fill in it's location, don't do that for the 4th command though. Also note you may have some certificates already on your system, so after the second last command you may be prompted to replace existing certificates, type N so it doesn't replace the ones you have:

$ sudo mkdir /etc/postfix/certs
$ sudo cp roots.zip /etc/postfix/certs
$ cd /etc/postfix/certs/
$ sudo unzip -j roots.zip
$ sudo openssl x509 -inform der -in thawte\ Primary\ Root\ CA\ -\ G2_ECC.cer -out thawte\ Primary\ Root\ CA\ -\ G2_ECC.pem
$ sudo c_rehash /etc/postfix/certs

Now type in:

$ sudo nano /etc/postfix/main.cf

Go to the end of the document, you can delete the MAMP stuff, also note that if you have MAMP Pro and you edit postfix settings from there it'll fuck up what we're doing here. So remember this going into the future and don't do that.

Paste in the following at the end of the document - note: use the keyboard to get around the document, but use the mouse to right click and paste:

relayhost = smtp.gmail.com:587
# auth
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/relay_password
smtp_sasl_security_options = noanonymous

# tls
smtp_tls_security_level = may
smtp_tls_CApath = /etc/postfix/certs
smtp_tls_session_cache_database = btree:/etc/postfix/smtp_scache
smtp_tls_session_cache_timeout = 3600s
smtp_tls_loglevel = 1
tls_random_source = dev:/dev/urandom

Save it like we did before by pressing ctrl+o, then Enter, then ctrl+x.

Now type in:

$ sudo nano /etc/postfix/master.cf

You're now editing master.cf, this is a different file to main.cf we just pasted stuff into. There should be a table in here, find the line in the table that looks something like this:

#tlsmgr    fifo  -       -       n       -       1       tlsmgr

Make it look like this - note the comment is removed and fifo should be unix:

tlsmgr    unix  -       -       n       -       1       tlsmgr

Save it like the other times pressing ctrl+o, then Enter, then ctrl+x.

Ok, so at this point you can put the following into terminal and see that it works - put your email address in there twice:

printf "Subject: blah" | sendmail -f user@gmail.com user@gmail.com

Postfix is working now. good. You go back to your PHP application and test the mail() function again. If it works then you're done, but if not you panic. You start feeling really hungry. You know that Postfix is working but maybe PHP or Apache haven't gotten the message yet. ok. So you see in Console that sendmail is crashing, you open the crash report in Terminal and it tells you there's an incompatible version of libxml. It wants 10 and you have 9. You begin questioning if any of this is worth it and maybe you should just go sit in front of a tv and forget about doing anything meaningful with the rest of your life.

After almost installing XCode and registering as an apple developer so you can make and install the newest version of libxml, you wonder if maybe MAMP comes with libxml and find that yes it does. So instead of spending 2 hours upgrading the system libxml only to find it doesn't do anything you just upgrade to the latest version of MAMP (1.8 at the time of writing) and it works. What?? it works? really? yup. so what do you do now?

REJOICE! with lunch.

Some of this was scoured from random forums and blogs in the midst of complete frustration and combined into steps that actually work. A chunk however was taken from this post: http://dejan.ranisavljevic.com/2009/05/28/enable-postfix-with-relay-outbound-to-your-gmail-account-on-os-x-leopard/ so check them out.