Dynamic/Metaprogramming in ABAP

http://scn.sap.com/thread/1374403#7695723 http://scn.sap.com/thread/699312

Preferred SSLCipherSuite for mod_ssl

SSLCipherSuite DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:HIGH:!3DES:!ECDH:!SRP:!aNULL:!CAMELLIA:!PSK:!EXPORT:!eNULL

Provides:

DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(128) Mac=AEAD
DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(256) Mac=AEAD
DHE-RSA-AES128-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA256
DHE-RSA-AES256-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA256
DHE-RSA-AES128-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA1
DHE-RSA-AES256-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA1
AES128-GCM-SHA256       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(128) Mac=AEAD
AES256-GCM-SHA384       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(256) Mac=AEAD
AES128-SHA256           TLSv1.2 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA256
AES256-SHA256           TLSv1.2 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA256
AES128-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA1
AES256-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA1
DHE-DSS-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=DSS  Enc=AESGCM(256) Mac=AEAD
DHE-DSS-AES256-SHA256   TLSv1.2 Kx=DH       Au=DSS  Enc=AES(256)  Mac=SHA256
DHE-DSS-AES256-SHA      SSLv3 Kx=DH       Au=DSS  Enc=AES(256)  Mac=SHA1
DHE-DSS-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=DSS  Enc=AESGCM(128) Mac=AEAD
DHE-DSS-AES128-SHA256   TLSv1.2 Kx=DH       Au=DSS  Enc=AES(128)  Mac=SHA256
DHE-DSS-AES128-SHA      SSLv3 Kx=DH       Au=DSS  Enc=AES(128)  Mac=SHA1

This is a modification of the ciphersuite list from http://www.matthewgkeller.com/blog/2014/01/09/ecdhe-vs-dhe-in-the-new-world-order/comment-page-1/

Primarily we are looking to remove elliptical curve ciphers in favor of discrete log methods (Schneier, 2013) due to the uncertainty of NSA compromization of ECC. We also try to prioritize the remaining available ciphers by preferring GCM mode over CBC mode.

1. AES128 has better key schedule than AES256 (Schneier, 2013).
2. We support "SSLv3" ciphers because in OpenSSL, TLSv1.0 ciphers are classified as SSLv3 ciphers.

SAP Internal Order Settlement Receiver Field

The value of this field depends on a business logic ruleset (thus, SAP GUI will report it as a structured field) when you inspect it from Internal Order display/Settlement rule tab. Here is (part of) the ruleset for determining the Settlement Receiver value: (Pseudo-SQL, NOT ABAP)

CASE 
   WHEN COBRB.KONTY = 'CTR'
      THEN COBRB.KOSTL
   WHEN COBRB.KONTY = 'FXA'
      THEN COBRB.ANLN1
END

(FXA is fixed asset settlement).

h/t (http://scn.sap.com/thread/3416443)

Turn off the login menu in web2py

Use Case: when you do not need to manage users or access control in the web2py application, remove the login menu from the right side of the menubar: Remove instantiating Auth(db) in your model and remove all dependencies on the Auth object.

web2py admin behind Apache proxy

So I have this web2py configuration where Apache httpd SSL proxies URLs of the form https://server/web2py/ to http://localhost:8081/web2py (web2py Rocket):

RewriteEngine on

RewriteCond %{HTTPS} !=on
RewriteCond %{HTTP_HOST} !=localhost
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]

RewriteRule ^/webpy$ /web2py/ [R,L]

ProxyRequests off
ProxyPass /web2py/ http://localhost:8081/web2py/

<Location /web2py/>
    ProxyPassReverse http://localhost:8081/web2py/
</Location>

This includes a modified global routes.py:

default_application = 'init'    # ordinarily set in base routes.py
default_controller = 'default'  # ordinarily set in app-specific routes.py
default_function = 'index'      # ordinarily set in app-specific routes.py

BASE = '/web2py'

routes_in = (
    # do not reroute admin unless you want to disable it
    (BASE + '/admin', '/admin/default/index'),
    (BASE + '/admin/$anything', '/admin/$anything'),
    # do not reroute appadmin unless you want to disable it
    (BASE + '/$app/appadmin', '/$app/appadmin/index'),
    (BASE + '/$app/appadmin/$anything', '/$app/appadmin/$anything'),
    # do not reroute static files
    (BASE + '/$app/static/$anything', '/$app/static/$anything'),
    # reroute favicon and robots, use exable for lack of better choice
    ('/favicon.ico', '/examples/static/favicon.ico'),
    ('/robots.txt', '/examples/static/robots.txt'),
    # do other stuff
    ((r'.*http://otherdomain\.com.* (?P.*)', r'/app/ctr\g')),
    # remove the BASE prefix
    (BASE + '/$anything', '/$anything'),
)

routes_out = [(x, y) for (y, x) in routes_in]

logging = 'debug'

#fix ticket routing
error_message = '<html><body><h1>%s</h1></body></html>'
error_message_ticket = '<html><body><h1>Internal error</h1>Ticket issued: <a href="' + BASE + '/admin/default/ticket/%(ticket)s" target="_blank">%(ticket)s</a></body></html>'

def __routes_doctest():
    pass

if __name__ == '__main__':
    import doctest
    doctest.testmod()

But if you go to https://hostname/web2py/admin, it returns Admin is disabled because insecure channel. However, the channel IS secure since we are using SSL via the Apache. Offending Lines of code: applications/admin/models/access.py:

if request.is_https:
    session.secure()
elif not request.is_local and not DEMO_MODE:
    raise HTTP(200, T('Admin is disabled because insecure channel'))

According to https://groups.google.com/forum/#!searchin/web2py-developers/request.is_local/web2py-developers/kkBvSzX4wO8/Rjom8huf4yMJ , request.is_local is False behind the Apache proxy, so calling https://server/web2py/admin fails both request.is_https (since the proxy forwards to http://) and request.is_local. Commenting out this block causes login dialog to fail (for the same reasons). Thus the correct modification is to use request.is_local = True

ETL for Reading ACL Analytics Exchange Server Job Logs (with AX Exception integration)

So you want have a machine way of reading the Job logs from ACL Analytics Exchange Server 3 or 4? Here's a SQL that lets you find out what analytic was started when and if any results were pushed to AX Exception: (Connect to the PostgreSQL database holding the backend of AX, by default it was called AclAuditExchangeDB in AX 3).

select t1.starttime, t2.name as analytic_name, 
t4.name as analytic_project, t6.name as activity, 
t7.name as engagement, t3.resulttable, t3.destinationentity, 
t3.destinationanalytic 
from scriptjobs t1
left outer join
audititems t2
on t1.analyticid = t2.id
left outer join
scriptjobpublish t3
on t1.jobnumber = t3.jobnumber
left outer join
audititems t4
on t2.parentid = t4.id
left outer join
audititems t5
on t4.parentid = t5.id
left outer join
audititems t6
on t5.parentid = t6.id
left outer join
audititems t7
on t6.parentid = t7.id

where t1.starttime > '2014-10-01'
order by engagement, activity, t1.starttime

Stripping table name prefixes from SAP DirectLink fieldnames

So if you are using ACL DirectLink to do a full table extraction out of SAP, it will create an ACL Table whose fieldnames are prefixed with %TABLENAME%_, so for example BUKRS from BSAK ends up as BSAK_BUKRS. When you are using DirectLink as an ETL shim to export a flatfile extraction, you sometimes want the native field names instead, without prefixes (Careful: if you have specified a server side join in DirectLink, it will use the tablename prefix to distinguish between the same fieldname in two different tables from the resulting join, for example: BKPF_BUKRS, BSAK_BUKRS. Of course, server side joins are to be avoided when doing this type of dump anyway). From the ACL project, extract the table definition:

BSAK_MANDT     UNICODE     1   6   AS "Accounting: Secondary Index for Vendors (Cleared Items);Client"  
BSAK_BUKRS     UNICODE     7   8   AS "Accounting: Secondary Index for Vendors (Cleared Items);Company Code"  
BSAK_LIFNR     UNICODE    15  20   AS "Accounting: Secondary Index for Vendors (Cleared Items);Account Number of Vendor or Creditor"  
BSAK_UMSKS     UNICODE    35   2   AS "Accounting: Secondary Index for Vendors (Cleared Items);Special G/L Transaction Type"  
BSAK_UMSKZ     UNICODE    37   2   AS "Accounting: Secondary Index for Vendors (Cleared Items);Special G/L Indicator"  
BSAK_AUGDT     DATETIME   39  16   PICTURE "YYYYMMDD" AS "Accounting: Secondary Index for Vendors (Cleared Items);Clearing Date"  
BSAK_AUGBL     UNICODE    55  20   AS "Accounting: Secondary Index for Vendors (Cleared Items);Document Number of the Clearing Document"  
...
etc

$ awk '{print $1}' bsak_all.txt | while read line; do echo "DELETE FIELD " $(echo $line | sed 's/[^_]*_\(.*\)/\1/') " OK" && echo "DEFINE FIELD " $(echo $line | sed 's/[^_]*_\(.*\)/\1/') " COMPUTED " $line && echo; done

non-greedy matching in sed

If you want to match a prefix in a string and stop at the n'th occurence of a delimiter in that string then you need non-greedy matching. This is how you would do it if the regex engine you are using doesn't support it natively, do it by negative character class.

Example: given a URI http://service.domain.tld/path1/path2/path3 and you only want service.domain.tld by splitting on /, then:

$ echo "http://service.domain.tld/path1/path2/path3" | sed 's@http://\([^/]*\).*@\1@'
service.domain.tld

Now what happens if you need the suffix after the delimiter? Just change the capturing parens:

$ echo "foo_bar_baz" | sed 's/[^_]*_\(.*\)/\1/'
bar_baz

Courtesy of: http://stackoverflow.com/questions/1103149/non-greedy-regex-matching-in-sed

Find which SAP roles provide access to what T-Codes

A lot of times, your SAP superuser wants you to request the specific role (or a template user) for which to provision a new account or role assignment you requested. Best way to determine that role is to search roles which fulfill the TCODEs you require access to. To do this, make sure you can login with a user with a role that has access to SUIM.

SUIM -> Roles -> By Authorization Values

Set Authorization Object 1 to S_TCODE

Click Entry value

Specify the TCodes you want to look up roles for. Pay attention to the AND and OR boxes for multiple TCode criteria.

Auth.user itself is a DAL row object

So after we instantiate auth=Auth(db=db), when the user logs in, Auth will create a subclass of type <class 'gluon.dal.Row'> which stores the fields from auth_user table:

email

first_name

id

last_name

registration_id

registration_key

reset_password_key

username

Locking down a default web2py controller

Say you have a web2py app where you want the default controller default.py to be locked down to require a user to always be authenticated to access any of the functions in that controller.

Problem: by default, user():, which calls auth() to supply the default login form handler is in default.py so if use the typical method to force auth checking on the controller itself (as explained in https://groups.google.com/d/msg/web2py/ReznbEX0Mh0/CfyEF70TrG0J)

auth.requires_login()(lambda: None)()

def index():
    return dict()

def user():
    return dict(form=auth())

This will result in a redirection loop because the initial un-logged-in access to /app/(default/index) will result in a redirect (due to requires_login()) to /app/default/user/login and on every call to default will result in a redirect to default/user/login etc.

So,we have to move user() out of the way.

To do this, we have to tell Auth where to find user(). By default it looks in default controller, but this can be modified in the Auth instantiation in the model (models/db.py):

#original auth instance
#auth = Auth(db)

#redirected auth instance
auth = Auth(db=db, controller='login')

This tells Auth instance auth to look in controller login for the user() instead of default. We use keyword db for telling it which DAL instance to use now, since we have switched to kwargs. In controllers/login.py:

# coding: utf8

#here is the default action, redirect back to the default controller
def index():
    redirect(URL(c='default'))

def user():
    return dict(form=auth())

And don't forget to move views/default/user.html to views/login/user.html!

Web2py Auth User against LDAP (Active Directory)

In models/db.py, adapted from the example one from welcome:

#using the default DAL db. You can use pg if you want
db = DAL('sqlite://storage.sqlite',pool_size=1,check_reserved=['all'])

#store sessions in the db not on the filesystem
session.connect(request, response, db=db)

#default boilerplate from welcome
response.generic_patterns = ['*'] if request.is_local else []

#default boilerplate from welcome
from gluon.tools import Auth, Crud, Service, PluginManager, prettydate
auth = Auth(db)
crud, service, plugins = Crud(db), Service(), PluginManager()

#use username as the primary id, not email address
auth.define_tables(username=True, signature=False)

#do not create a default user group (=user) for every user that gets imported
auth.settings.create_user_groups=False

#default config from welcome
mail = auth.settings.mailer
mail.settings.server = 'logging' or 'smtp.gmail.com:587'
mail.settings.sender = 'you@gmail.com'
mail.settings.login = 'username:password'

#comment these from the default
#auth.settings.registration_requires_verification = False
#auth.settings.registration_requires_approval = False
#auth.settings.reset_password_requires_verification = True

#LDAP is always the system of record, so disable manual registration or the changing of the user in the app
auth.settings.actions_disabled=['register','change_password','request_reset_password','retrieve_username','profile']

#this is just good security
auth.settings.remember_me_form = False

#import ldap_auth method
from gluon.contrib.login_methods.ldap_auth import ldap_auth

#override all/any default auth settings, users can *only* auth against Active Directory
auth.settings.login_methods=[ldap_auth(mode='ad',
                                       manage_user=True,
                                       user_firstname_attrib = 'givenName',
                                       user_lastname_attrib = 'sn',
                                       user_mail_attrib = 'mail',
                                       server='corp.contoso.com',
                                       base_dn='dc=contoso,dc=com',
                                       secure=True,
                                       db=db)]

#disable janrain
#from gluon.contrib.login_methods.rpx_account import use_janrain
#use_janrain(auth, filename='private/janrain.key')

Usage: The Login dialog will cause this web2py app to autocreate a user based on the attributes in LDAP. The actual auth is the return of a successful LDAP bind. You can also pre-create users using appadmin. When manually creating users this way, you will need to set a dummy password in the db since it is set to be a required field (but will remain empty when the user is autocreated...). You may want to manually add users when you are setting up app-specific groups.

Caveats: To get LDAP secure=True working with a self-signed cert on the webserver, I had to hack gluon/contrib/login_methods/ldap_auth.py: In ldap_auth().init_ldap(), I had to add the following after if secure:

ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_ALLOW)

See also: http://www.web2pyslices.com/slice/show/1715/authentication-and-group-control-with-active-directory-ldap if you want to base RBAC off AD groups.

Testing prettyprint

I edited my Blogger template to include the following inside <head> tags: (h/t Pacini's Space)

<script src="https://c328740.ssl.cf1.rackcdn.com/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript">
</script>
<script src="https://google-code-prettify.googlecode.com/svn/loader/run_prettify.js?lang=tex&skin=sunburst">
</script>

import cow
mycow = cow.create(name='cowbert')
output = []
for i in xrange(10):
    output.append(mycow.emit('moo'))

print ' '.join(output)

UPDATE 2016-08-03:

See https://github.com/google/code-prettify

Hello World

So I figured this was a better place to put related work/dev stuff instead of doing it on G+. I'll be copying relevant full G+ posts here and from here on out post here first, then link to the post on G+ when I want to publish.