Stripping table name prefixes from SAP DirectLink fieldnames

So if you are using ACL DirectLink to do a full table extraction out of SAP, it will create an ACL Table whose fieldnames are prefixed with %TABLENAME%_, so for example BUKRS from BSAK ends up as BSAK_BUKRS. When you are using DirectLink as an ETL shim to export a flatfile extraction, you sometimes want the native field names instead, without prefixes (Careful: if you have specified a server side join in DirectLink, it will use the tablename prefix to distinguish between the same fieldname in two different tables from the resulting join, for example: BKPF_BUKRS, BSAK_BUKRS. Of course, server side joins are to be avoided when doing this type of dump anyway). From the ACL project, extract the table definition:

BSAK_MANDT     UNICODE     1   6   AS "Accounting: Secondary Index for Vendors (Cleared Items);Client"  
BSAK_BUKRS     UNICODE     7   8   AS "Accounting: Secondary Index for Vendors (Cleared Items);Company Code"  
BSAK_LIFNR     UNICODE    15  20   AS "Accounting: Secondary Index for Vendors (Cleared Items);Account Number of Vendor or Creditor"  
BSAK_UMSKS     UNICODE    35   2   AS "Accounting: Secondary Index for Vendors (Cleared Items);Special G/L Transaction Type"  
BSAK_UMSKZ     UNICODE    37   2   AS "Accounting: Secondary Index for Vendors (Cleared Items);Special G/L Indicator"  
BSAK_AUGDT     DATETIME   39  16   PICTURE "YYYYMMDD" AS "Accounting: Secondary Index for Vendors (Cleared Items);Clearing Date"  
BSAK_AUGBL     UNICODE    55  20   AS "Accounting: Secondary Index for Vendors (Cleared Items);Document Number of the Clearing Document"  
...
etc

$ awk '{print $1}' bsak_all.txt | while read line; do echo "DELETE FIELD " $(echo $line | sed 's/[^_]*_\(.*\)/\1/') " OK" && echo "DEFINE FIELD " $(echo $line | sed 's/[^_]*_\(.*\)/\1/') " COMPUTED " $line && echo; done

non-greedy matching in sed

If you want to match a prefix in a string and stop at the n'th occurence of a delimiter in that string then you need non-greedy matching. This is how you would do it if the regex engine you are using doesn't support it natively, do it by negative character class.

Example: given a URI http://service.domain.tld/path1/path2/path3 and you only want service.domain.tld by splitting on /, then:

$ echo "http://service.domain.tld/path1/path2/path3" | sed 's@http://\([^/]*\).*@\1@'
service.domain.tld

Now what happens if you need the suffix after the delimiter? Just change the capturing parens:

$ echo "foo_bar_baz" | sed 's/[^_]*_\(.*\)/\1/'
bar_baz

Courtesy of: http://stackoverflow.com/questions/1103149/non-greedy-regex-matching-in-sed

Find which SAP roles provide access to what T-Codes

A lot of times, your SAP superuser wants you to request the specific role (or a template user) for which to provision a new account or role assignment you requested. Best way to determine that role is to search roles which fulfill the TCODEs you require access to. To do this, make sure you can login with a user with a role that has access to SUIM.

SUIM -> Roles -> By Authorization Values

Set Authorization Object 1 to S_TCODE

Click Entry value

Specify the TCodes you want to look up roles for. Pay attention to the AND and OR boxes for multiple TCode criteria.

Auth.user itself is a DAL row object

So after we instantiate auth=Auth(db=db), when the user logs in, Auth will create a subclass of type <class 'gluon.dal.Row'> which stores the fields from auth_user table:

email

first_name

id

last_name

registration_id

registration_key

reset_password_key

username

Locking down a default web2py controller

Say you have a web2py app where you want the default controller default.py to be locked down to require a user to always be authenticated to access any of the functions in that controller.

Problem: by default, user():, which calls auth() to supply the default login form handler is in default.py so if use the typical method to force auth checking on the controller itself (as explained in https://groups.google.com/d/msg/web2py/ReznbEX0Mh0/CfyEF70TrG0J)

auth.requires_login()(lambda: None)()

def index():
    return dict()

def user():
    return dict(form=auth())

This will result in a redirection loop because the initial un-logged-in access to /app/(default/index) will result in a redirect (due to requires_login()) to /app/default/user/login and on every call to default will result in a redirect to default/user/login etc.

So,we have to move user() out of the way.

To do this, we have to tell Auth where to find user(). By default it looks in default controller, but this can be modified in the Auth instantiation in the model (models/db.py):

#original auth instance
#auth = Auth(db)

#redirected auth instance
auth = Auth(db=db, controller='login')

This tells Auth instance auth to look in controller login for the user() instead of default. We use keyword db for telling it which DAL instance to use now, since we have switched to kwargs. In controllers/login.py:

# coding: utf8

#here is the default action, redirect back to the default controller
def index():
    redirect(URL(c='default'))

def user():
    return dict(form=auth())

And don't forget to move views/default/user.html to views/login/user.html!

Web2py Auth User against LDAP (Active Directory)

In models/db.py, adapted from the example one from welcome:

#using the default DAL db. You can use pg if you want
db = DAL('sqlite://storage.sqlite',pool_size=1,check_reserved=['all'])

#store sessions in the db not on the filesystem
session.connect(request, response, db=db)

#default boilerplate from welcome
response.generic_patterns = ['*'] if request.is_local else []

#default boilerplate from welcome
from gluon.tools import Auth, Crud, Service, PluginManager, prettydate
auth = Auth(db)
crud, service, plugins = Crud(db), Service(), PluginManager()

#use username as the primary id, not email address
auth.define_tables(username=True, signature=False)

#do not create a default user group (=user) for every user that gets imported
auth.settings.create_user_groups=False

#default config from welcome
mail = auth.settings.mailer
mail.settings.server = 'logging' or 'smtp.gmail.com:587'
mail.settings.sender = 'you@gmail.com'
mail.settings.login = 'username:password'

#comment these from the default
#auth.settings.registration_requires_verification = False
#auth.settings.registration_requires_approval = False
#auth.settings.reset_password_requires_verification = True

#LDAP is always the system of record, so disable manual registration or the changing of the user in the app
auth.settings.actions_disabled=['register','change_password','request_reset_password','retrieve_username','profile']

#this is just good security
auth.settings.remember_me_form = False

#import ldap_auth method
from gluon.contrib.login_methods.ldap_auth import ldap_auth

#override all/any default auth settings, users can *only* auth against Active Directory
auth.settings.login_methods=[ldap_auth(mode='ad',
                                       manage_user=True,
                                       user_firstname_attrib = 'givenName',
                                       user_lastname_attrib = 'sn',
                                       user_mail_attrib = 'mail',
                                       server='corp.contoso.com',
                                       base_dn='dc=contoso,dc=com',
                                       secure=True,
                                       db=db)]

#disable janrain
#from gluon.contrib.login_methods.rpx_account import use_janrain
#use_janrain(auth, filename='private/janrain.key')

Usage: The Login dialog will cause this web2py app to autocreate a user based on the attributes in LDAP. The actual auth is the return of a successful LDAP bind. You can also pre-create users using appadmin. When manually creating users this way, you will need to set a dummy password in the db since it is set to be a required field (but will remain empty when the user is autocreated...). You may want to manually add users when you are setting up app-specific groups.

Caveats: To get LDAP secure=True working with a self-signed cert on the webserver, I had to hack gluon/contrib/login_methods/ldap_auth.py: In ldap_auth().init_ldap(), I had to add the following after if secure:

ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_ALLOW)

See also: http://www.web2pyslices.com/slice/show/1715/authentication-and-group-control-with-active-directory-ldap if you want to base RBAC off AD groups.

Testing prettyprint

I edited my Blogger template to include the following inside <head> tags: (h/t Pacini's Space)

<script src="https://c328740.ssl.cf1.rackcdn.com/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript">
</script>
<script src="https://google-code-prettify.googlecode.com/svn/loader/run_prettify.js?lang=tex&skin=sunburst">
</script>

import cow
mycow = cow.create(name='cowbert')
output = []
for i in xrange(10):
    output.append(mycow.emit('moo'))

print ' '.join(output)

UPDATE 2016-08-03:

See https://github.com/google/code-prettify