|
Tutorial: Create an NFS-like Storage Server with GlusterFS on Ubuntu 12.10 |
vote
 |
|
| This tutorial shows how to set up a standalone storage server on Ubuntu 12.10. Instead of NFS, I will use GlusterFS here. The client system will be able to access the storage as if it was a local filesystem. GlusterFS is a clustered file-system capable of scaling to several peta-bytes. It aggregates various storage bricks over Infiniband RDMA or TCP/IP interconnect into one large parallel network file system. Storage bricks can be made of any commodity hardware such as x86_64 servers with SATA-II RAID and Infiniband HBA. read more... |
|
| | mail this link | permapage | score:9614 | -falko, January 16, 2013 |
|
OpenShift Online: a non-developer guide |
vote
|
|
| OpenShift Online is a developer-targeted service, but this article is aimed at non-developers, just to highlight the aspect of the service you can take advantage of. read more... |
|
| | permapage | score:9483 | -finid, March 31, 2013 |
|
Build Apps with Android SDK, Eclipse, PhoneGap (Ubuntu 10.10) |
vote
|
|
| This tutorial describes how you can set up a development environment for building Android apps on an Ubuntu 10.10 desktop using Eclipse, the Android SDK, and PhoneGap. I will describe how to build Android apps from the command line with PhoneGap and from the GUI with Eclipse and PhoneGap and how to test them in an Android emulator and on a real Android device. PhoneGap allows you to develop your Android applications using web technologies such as HTML, CSS, and JavaScript (e.g. with JavaScript libraries such as jQuery/jQTouch), and it will turn these web apps into native Android apps (in fact, PhoneGap supports multiple platforms such as Android, iPhone, Palm, Windows Mobile, Symbian, so you can use the same sources to create apps for multiple platforms). read more... |
|
| | mail this link | permapage | score:9446 | -falko, January 27, 2011 |
|
e-book: Sed One-Liners Explained |
vote
|
|
I love writing about programming and I am happy to announce my second e-book called "Sed One-Liners Explained".
Sed one-liners are short sed scripts for everyday situations in the shell, such as changing line spacing, numbering lines, and converting and deleting text.
For example, the following sed one-liner numbers the lines of a file:
sed = file | sed 'N; s/n/: /'
Here is how it works - it's made out of two sed commands. The first one uses the = command that inserts a line containing the line number before every original line in the file. Then this output gets piped to the second sed command that joins two adjacent lines with the N command. When joining lines with the N command, a newline character n is placed between them. Therefore it uses the s command to replace this newline n with a colon followed by a space ": ".
The e-book is 98 pages long and it explains exactly 100 one-liners. It's divided into the following chapters:
Preface.
1. Introduction to sed.
2. Line Spacing.
3. Line Numbering.
4. Text Conversion and Substitution.
5. Selective Printing of Certain Lines.
6. Selective Deletion of Certain Lines.
7. Special sed Applications.
Appendix A. Summary of All sed Commands.
Appendix B. Addresses and Ranges.
Appendix C. Debugging sed Scripts with sed-sed.
Index.
Did you know that sed was as powerful as any other programming language? Someone even wrote Tetris in it.
After you read the e-book, you'll be able to write your own Tetris if you wanted to. read more... |
|
| | mail this link | permapage | score:9372 | -pkrumins, September 19, 2011 |
|
Expect Script Examples |
vote
|
|
Expressions, if statements, for loops, and while loops examples are covered in this mini-tutorial:
This article explains the following in the expect scripting language.- Expressions – arithmetic operation
- if construct in expect
- looping constructs
read more... |
|
| | permapage | score:9363 | -Ray, January 21, 2011 |
|
Develop your own Raspberry Pi OS |
vote
|
|
| Why use an existing operating system if you can bake your own? Well, for one thing, writing an OS is not an easy feat. It requires a lot of knowledge in a lot of areas regarding OS development. read more... |
|
| | permapage | score:9316 | -finid, September 7, 2012 |
|
Tutorial: Android app build environment with Eclipse, PhoneGap (Ubuntu 11.04) |
vote
|
|
| This tutorial describes how you can set up an development environment for building Android apps on an Ubuntu 11.04 desktop using Eclipse, the Android SDK, and PhoneGap. I will describe how to build Android apps from the command line with PhoneGap and from the GUI with Eclipse and PhoneGap and how to test them in an Android emulator and on a real Android device. PhoneGap allows you to develop your Android applications using web technologies such as HTML, CSS, and JavaScript (e.g. with JavaScript libraries such as jQuery/jQTouch), and it will turn these web apps into native Android apps (in fact, PhoneGap supports multiple platforms such as Android, iPhone, Palm, Windows Mobile, Symbian, so you can use the same sources to create apps for multiple platforms). read more... |
|
| | mail this link | permapage | score:9290 | -falko, June 28, 2011 |
|
Tutorial: Install and test C, C++ Compilers in Ubuntu |
vote
|
|
| If you are a developer you need a C and C++ Compiler for your development work on your Ubuntu system this tutorial will explain howto install C and C++ Compilers in Ubuntu with program examples. In ubuntu you can install the build-essential for C and C++ compilers. read more... |
|
| | permapage | score:9268 | -gg234, May 6, 2008 |
|
DeviceAnywhere: testing mobile apps |
vote
|
|
| There isn't just one mobile space, but literally hundreds split across multiple operating systems, devices and networks. Learn how DeviceAnywhere testing can help you deploy your Smartphone app across all the multitude of devices and their distinct flavor. read more... |
|
| | permapage | score:9264 | -solrac, January 13, 2011 |
|
Add Video Streaming to C/C++ apps with Nex Gen Media Server API |
vote
|
|
| Recently I took a closer look at Nex Gen Media Server and their API framework. NGMS is a multi-purpose streaming server which supports some of the popular streaming protocols such as RTSP, RTMP, Apple's HTTP Live, and MPEG-2 Transport Stream. NGMS comes with transcoding support and is able to capture and reformat live video streams and adapt them to be received by another type of device, such as capturing an HD video feed and converting it to be received by an iPhone over 3g. My focus was to integrate the NGMS API to control the streaming features directly from my own C application. In this example I am using Ubuntu Linux 10.04. read more... |
|
| | mail this link | permapage | score:9262 | -falko, November 17, 2011 |
|
String matching in regular expressions |
vote
|
|
| Use parentheses to create the string matches you need in regular expressions. Parentheses allows you to use pipes for multiple matches. read more... |
|
| | permapage | score:9247 | -aweber, September 12, 2011 |
|
Apache2, mod_rewrite tutorial: Redirect requests by device |
vote
|
|
| Since the massive rise of smartphones and tablets like the iPhone, iPad, Android phones and tablets, BlackBerries, etc. you might have considered creating a mobile version of your web site. This tutorial explains how to configure Apache to serve the mobile version of your web site if the visitor uses a mobile device, and the normal version if the visitor uses a normal desktop PC. This can be achieved with Apache's rewrite module. read more... |
|
| | permapage | score:9230 | -falko, September 9, 2011 |
|
Tutorial: Linux game programming with Ogre 3D |
vote
|
|
This tutorial starts at the beginning with opening a window...
This tutorial series steps you through the process of creating a 3D shoot'em'up game using the popular and powerful Ogre 3D engine. The tutorials compile on both Windows and Linux. read more... |
|
| | permapage | score:9224 | -Ray, January 1, 2010 |
|
Space Tyrant: Multithreading lessons learned on SMP hardware |
vote
|
|
There is much to report in this update of Space Tyrant. Before getting into the new features and functions, I’ll dispense with the crisis of The Bug.
For a couple of weeks, we had been noticing odd anomalies with Space Tyrant (ST) running on the virtual server at Ioresort.com (now offline -Ed.). We never saw the problem on any other box -- and it was tested on at least four other Linux boxes and a Mac OS X system. We did all manner of stress testing, locally and over the Internet, script based and even feeding the game the output of /dev/random. Nothing caused the anomaly on any other box.
At first, I suspected that it might just be an obscure problem with the virtual server itself; after all, I had been forced to modify the TLR code to get it to run properly there. That problem turned out to be merely a limitation of NFS, not a bug with the virtual server software. However, the environment was clearly different from any other system I had used which raised my suspicions -- and reduced my urgency about looking for the bug.
While the bug wasn’t frequent, it was persistent. The bug appeared to be related to corrupted buffers or corrupted buffer indexes. Out of idle curiosity, I lowered the number of buffers used by ST to see if that affected the bug. Somewhat counter-intuitively, it substantially raised the frequency of the problem.
Brian Estabrooks (the hero of this release) and I spent more and more of our efforts hunting this incredibly elusive bug until that was all we were doing. I implemented various diagnostic routines hunting for clues. The all seemed to point to buffer indexes being changed incorrectly. Both Brian and I audited the code. It seemed impossible for the indexes to be changed improperly. Brian even went so far as to replace the ring buffer scheme with a high watermark approach but to no avail.
While I continued to suspect it to be a simple logic error in the code, Brian turned his efforts elsewhere. What he came up with was quite interesting. It seems that on many hardware architectures (most? all?), modifying a bit field can temporarily modify other bit fields in the same word! Now, this isn’t a problem on a single-CPU system; it repairs the damage in the same operation, making it, effectively, atomic. On an SMP machine, however, two different CPU’s working on different bit fields of the same word simultaneously create havoc. The operation isn’t really atomic and it doesn’t work.
Did I mention that the virtual server is a 4-way Xeon system?
The ring buffer indexing in ST relies on unsigned integer bit fields to automate wrapping back around to the first buffer after using the last one. My parsimonious programming, of course, packed all the bit fields together, several to a word. Brian’s test version of ST added a pad after each buffer index to round it out so that each bit field lived alone in its own complete word. We abused the new version for nearly an hour before either of us would dare say it. The bug was gone.
Yay!
So, the moral of this story is: Operations on sub-word fields affect other bits in that word (at least on many hardware architectures). Tread very carefully if multiple threads are accessing different bits in shared words. It may appear to work perfectly, only to crumble into a pile of smoldering rubble the first time it's loaded on a multiple CPU system!
Other than the primary lesson, some other good things came out of (the search for) the bug. Several other latent bugs were found and fixed and Brian and I are both much more intimate with the code.
And, on to the enhancements. ST is starting to look like an actual playable game. The following functions implement the new major features.
players(): We now have player rankings. It works by adding all the players’ ship resources to an integer array. Then it scans the universe looking for deployed fighters and adds those to the array as well. Currently, those two items comprise the total strength of a player.
It then sorts the array with a recursive bit-plane sort that I wrote for Starship Traders in 1998. The qsort() function in the C library was plenty fast, but took too much memory for my taste. Memory was a bit scarcer in those days and, worse, the SST software model gave each player his own copy of the server.
The sort reorders the array in place as follows. It scans the high-order bit in each element of the array. It then moves all elements starting with ‘1’ bits to the top and all starting with ‘0’ bits to the bottom. Next, it calls itself twice to reorder the first and second chunks of the array on the second bit. Each of those two instances of the sort then call the sort twice again, now giving 4 new sorts for the third bit, and so on. When all 32 bits are accounted for, the array is in the correct order with the top player on top, etc.
Scanning the entire universe can be expensive with a large map. Therefore, the player rankings function keeps the result and time stamps it. If another player asks for a player ranking within five seconds, the system just gives them the old one. After five seconds, however, any new request triggers a fresh listing.
autopilot(): We’ve added an autopilot to let a player find a specific sector -- or to locate the nearest planet. If you type a ‘0’ (zero), you’ll be prompted for a sector number within 1000 of the sector you’re currently in. You then will have the option of pressing ‘/’ to automatically warp to the destination sector.
If you’re looking for a planet, type the ‘L’ command that you would normally use to land on a planet in your sector. In the absence of a planet, the L key will engage the autopilot which will search for the nearest planet and give you a ‘/’ command to autowarp to it.
The new autopilot function consists of two other functions in addition to autopilot(), which is merely a control function. I had intended to use the old shortest path algorithm function from TLR but it was big and complicated. I decided to try to write a simpler, recursive shortest path algorithm instead. The new recursive function is much simpler but not quite as efficient as the giant for loop in TLR.
The actual algorithm is implemented in two functions called pathdepth() and pathcalc(). The pathdepth() function repeatedly calls pathcalc() with an increasing ‘depth’ parameter. ‘Depth’ tells pathcalc() how many levels deep to search before giving up.
The pathcalc() function simply looks to see if the sector it is looking at is the target sector. If not, it calls itself for each new sector that the current sector connects to. If the current sector is the target sector, it starts filling in an array for the autowarp() function to follow to reach the target sector. As the previous recursive calls to the pathcalc() function exit, they fill in the remainder of the path array.
And, yes, I seem to like reinventing the wheel. ;-)
The other interesting addition to the code is the backup thread. It is implemented by a function called backupdata() and works as follows: It scans the player data, the map data, and the history data looking for ‘dirty’ flags. (Whenever any persistent data is changed anywhere in the game, a dirty flag is set to tell the backup thread to write it out to disk.) This process is quite fast for a small game, but for a game with millions of sectors, it’s a significant waste of resources to scan the dirty flag array frequently.
Therefore, for the map and history data, I’ve implemented a ‘dirty block’ scheme as well. When a dirty flag is set, its corresponding dirty block flag is set too. Then, the backup thread need only scan the dirty block arrays, typically only about one percent the size of the arrays it represents. When a dirty block is found, only the hundred or so records it points to are scanned to find the actual dirty records for backup.
The backup file, named ‘st.9999.dat’ -- where ‘9999’ varies with the port number you run the game on -- goes into the current working directory from where you start the daemon. If the file doesn’t exist, a new game is started. Also, if you’ve modified the game in a way that changes the size of the data -- by increasing the map size, for example -- it will start a new game upon startup.
The game can be shut down from the command line by sending a signal 15 (kill -15 pid) or by the admin with the ^ command. Note that the first player to create an account in a new game automatically becomes the admin of the game!
makehistory(): The storing of historical data is new as well. Whenever another player attacks your ship while you’re logged off, you’ll get a report of the action and any losses when you next log on. Also, for remote deployed fighters, you never get immediate notification, so that information is stored in the history log even if you're logged on when it happens. You can view any accumulated event information since your login time by pressing the ‘e’ key.
deploy(): This simple function allows a player to deploy, or retrieve, guard fighters in a sector. Those fighters will not let another player pass through or view any of the contents of that sector. Any ships parked under the fighters are automatically protected against all attacks except for an attack by the fighters’ owner. Once the fighters are destroyed, of course, all ships there are visible and can be attacked.
There is also a newly implemented time limit in the game to limit the total online time of a day’s sessions to 4 hours. Like most other parameters, it can be changed by modifying a #define statement near the top of the code.
command(): The help page, a menu of available commands that a player can perform, has been redesigned and rewritten. This menu is attached to the '?' key.
The old debugger thread is gone, replaced by an in-game command function called showdata(). Press the ‘z’ key to see information on buffers, buffer indexes, and the backup thread’s state and history. Only if you’re serious about modifying the code will this information be useful.
The section of the gameloop thread that broadcasts radio and news messages has been modified to show only one of each type of message per pass. That way, replaying a long radio history won’t flood the output buffers and longer radio and news histories can therefore be retained.
The old jumprtn() movement function has been consolidated into the warprtn() function. It’s only slightly more complicated than having them separate.
The current source code can be downloaded from http://librenix.com/st/st.158.c. and the original article in this series is here. As usual, the compile script is embedded in the comments at the top of the source file. You’ll have to rename the source st.c for the script to work unchanged.
[A Space Tyrant home page has been created as a central index to the various ST articles, links, and files.] |
|
| | mail this link | permapage | score:9205 | -Ray, June 26, 2005 (Updated: July 26, 2008) |
|
Scripting: Put a clock in your bash terminal |
vote
|
|
In the original version, the cursor positioning didn't work on my Mac OS X system. If that happens to you, try this simplified variant: #!/bin/bash
while true
do
tput sc
tput cup 0 60
echo -en `date +"%H:%M:%S %F"`
tput rc
sleep 1
done
Also, note that you'll need to run either script in the background to use your terminal.
The script saves the current cursor position with an ANSI escape sequence instruction. Then, using the tput command, the cursor is sent to row 0 (the top of the screen) and the last column minus 19 characters (19 is the length of HH:MM:SS YYYY-MM-DD). The formatted date command is displayed in green inverted color. The cursor is then sent back to its original position with another ANSI sequence that restores the original saved position. read more... |
|
| | mail this link | permapage | score:9183 | -Ray, January 22, 2008 |
|
Create a UI fast |
vote
|
|
Create a nice User Interface using Qt Designer in a couple of minutes!We will create a mini-calculator that, by the end of this tutorial, should look like this:
[...]
As usual, I'll be using a ubuntu box, as well as Qt 4.7. Qt is cross-platform, so don't worry if you're using another Linux flavor or Windows: the steps are almost the same :) read more... |
|
| | permapage | score:9173 | -aimar, January 3, 2011 |
|
Space Tyrant: A threaded game server project in C |
vote
|
|
[Update, June 25, 2005: A Space Tyrant home page has been created as a central index to the various ST articles, links, and files.]
[Update, March 21, 2007: A Space Tyrant has its own website! It's small but growing and will provide quick access to the latest code and developments in the ST universe.]
Space Tyrant: Today we kick off a new multithreaded, network socket programming project which we will call Space Tyrant. Our mission is to write an open source, multiplayer, networked, strategy game in the C programming language. The goal of this project is to make a solid code base which implements a simple space trading game upon which other games can then be built. The game will be a subset of The Last Resort (TLR) that currently runs at [offline]. This project will be a learning exercise for me as well as for any interested readers. The current state of the source code will be released with each article update.
The game design: While my TLR game consists of over 25,000 lines of C source code and supports a web interface as well as telnet and a graphical client, this code will be far smaller and simpler. It will initially only support telnet and will implement a far simpler game design.
Players will be able to telnet into the game, create an account, and play in a universe that contains ports, planets, as well as other players. Each player will be issued a starship, some cargo holds, and an amount of starship fuel. Additional fuel will be issued hourly and will accumulate in the starship. Fuel will be used to move the ship between sectors -- locations within the game universe -- and to dock with ports. Once a ship runs out of fuel it can't move at all until new fuel is issued.
Players will be able to buy and sell commodities (Iron, alcohol, and hardware) between the three different kinds of ports. Each port type will sell one of the three commodities and buy the other two. Prices will be based on supply and demand with rarely-used ports offering the better prices.
With the money players earn trading they will be able to buy more cargo holds to make their ships more efficient for trading. They will also be able to buy fighters -- small military drones -- that can be used to attack other ships or deployed to guard a sector and its contents. The fighters carried with a ship will guard it against attacks from other players.
Games will run for a predetermined length of time, then reset and start anew.
The programming model: Now, on to the software design. I've compared and considered various models for the server design. TLR is based on the forking model using inetd or xinetd to handle the listening and forking. While the forking model is inherently distributable to multiple processors, it introduces inefficiencies (forking multiple processes) and makes interprocess communications more difficult and slower.
Next, I considered a non-blocking, single process model. In this approach, one process handles everything in a single thread. It would use non-blocking IO (read and write functions that never wait for completion but, rather, return immediately if they aren't ready to read or write actual data). The thttpd web server is an example of a non-blocking, single process server. It's extremely fast and efficient. However, this model is quite complicated to code, and, I believe would make it more likely to introduce subtle timing bugs.
Next, I considered a pure multithreaded, single process model with a thread for each player. While appealing in many ways, this model would require the same kind of coordination between threads that the forking model requires between processes. Such interprocess communication would be simplified in that the various threads share memory, but the coordination issues otherwise remain the same.
Last, I considered another multithreaded model, this time with only IO threads for each user and a single thread that implements all game logic. While that one central thread might someday be a bottleneck that limits scalability on large SMP systems, it does distribute the IO on any additional processors that might be present, and requires minimal coordination. In short, this model combines the logic simplicity of the non-blocking single process model with the coding simplicity of the threaded model, while separating the IO from the main logic. There will also be two other simple threads in this model. There will be a thread that listens for new connections and spawns the IO threads for each new connection. There will also be a thread that writes the data to disk periodically.
This is the approach that I intend to take for this project. The code will be written for both Linux and Mac OS X.
More info: I have set up an email address for programmers following this series to provide recommendations, bug reports, and other feedback. Send email about this project to spacetyrant [at] librenix.com. |
|
| | mail this link | permapage | score:9154 | -Ray, March 18, 2005 (Updated: July 26, 2008) |
|
Detailed Error Handling In Bash |
vote
|
|
| Shell scripts are often running as background processes, doing useful things without running in a visible shell. To write such scripts can be quite painful, as all errors occur out of sight as well. While log files can hold a lot of information, finding the relevant information is a bit trickier. My solution is to log only the errors with all the details to a small database. This database contains tables for the message, the corresponding stack trace and the important environment variables. I have chosen for an SQLite database in this howto, but the same principle works with other databases as well. read more... |
|
| | mail this link | permapage | score:9139 | -falko, February 21, 2013 |
|
Using Git for Source Control |
vote
|
|
| GIT is the source control tool, a distributed version control system (dvcs) which is written in C. This provides a history of the files that are maintained by it. In distributed version control each user has a complete copy of the code so there is no central code repository. In this scenario an administrator makes changes, adds them to the index (called staging) and then adds them to the repository (called commit). Git will take this information and maintain a version history that users can track. This is all performed locally but could be synchronized with a remote repository. read more... |
|
| | mail this link | permapage | score:9138 | -aweber, May 5, 2012 |
|
Sustitution with sed |
vote
|
|
| The Linux utility sed provides a great way to substitute text strings in a file. Using the "s" option and by listing the current string and the string to use as the new text allows sed to perform this task. read more... |
|
| | permapage | score:9137 | -aweber, August 3, 2011 |
|
|
